Version: dev

Field Configuration

This page explains how to configure field-level metadata for serialization.

Overview

Apache Fory™ provides two ways to specify field-level metadata at compile time:

fory::field<> template - Inline metadata in struct definition
FORY_FIELD_TAGS macro - Non-invasive metadata added separately

These enable:

Tag IDs: Assign compact numeric IDs for schema evolution
Nullability: Mark pointer fields as nullable
Reference Tracking: Enable reference tracking for shared pointers

The fory::field Template

template <typename T, int16_t Id, typename... Options>
class field;

Template Parameters

Parameter	Description
`T`	The underlying field type
`Id`	Field tag ID (int16_t) for compact serialization
`Options`	Optional tags: `fory::nullable`, `fory::ref`

Basic Usage

#include "fory/serialization/fory.h"

using namespace fory::serialization;

struct Person {
  fory::field<std::string, 0> name;
  fory::field<int32_t, 1> age;
  fory::field<std::optional<std::string>, 2> nickname;
};
FORY_STRUCT(Person, name, age, nickname);

The fory::field<> wrapper is transparent - you can use it like the underlying type:

Person person;
person.name = "Alice";           // Direct assignment
person.age = 30;
std::string n = person.name;     // Implicit conversion
int a = person.age.get();        // Explicit get()

Tag Types

fory::nullable

Marks a smart pointer field as nullable (can be nullptr):

struct Node {
  fory::field<std::string, 0> name;
  fory::field<std::shared_ptr<Node>, 1, fory::nullable> next;  // Can be nullptr
};
FORY_STRUCT(Node, name, next);

Valid for: std::shared_ptr<T>, std::unique_ptr<T>

Note: For nullable primitives or strings, use std::optional<T> instead:

// Correct: use std::optional for nullable primitives
fory::field<std::optional<int32_t>, 0> optional_value;

// Wrong: nullable is not allowed for primitives
// fory::field<int32_t, 0, fory::nullable> value;  // Compile error!

fory::not_null

Explicitly marks a pointer field as non-nullable. This is the default for smart pointers, but can be used for documentation:

fory::field<std::shared_ptr<Data>, 0, fory::not_null> data;  // Must not be nullptr

Valid for: std::shared_ptr<T>, std::unique_ptr<T>

fory::ref

Enables reference tracking for shared pointer fields. When multiple fields reference the same object, it will be serialized once and shared:

struct Graph {
  fory::field<std::string, 0> name;
  fory::field<std::shared_ptr<Graph>, 1, fory::ref> left;    // Ref tracked
  fory::field<std::shared_ptr<Graph>, 2, fory::ref> right;   // Ref tracked
};
FORY_STRUCT(Graph, name, left, right);

Valid for: std::shared_ptr<T> only (requires shared ownership)

fory::dynamic<V>

Controls whether type info is written for polymorphic smart pointer fields:

fory::dynamic<true>: Force type info to be written (enable runtime subtype support)
fory::dynamic<false>: Skip type info (use declared type directly, no dynamic dispatch)

By default, Fory auto-detects polymorphism via std::is_polymorphic<T>. Use this tag to override:

// Base class with virtual methods (detected as polymorphic by default)
struct Animal {
  virtual ~Animal() = default;
  virtual std::string speak() const = 0;
};

struct Zoo {
  // Auto: type info written because Animal has virtual methods
  fory::field<std::shared_ptr<Animal>, 0, fory::nullable> animal;

  // Force non-dynamic: skip type info even though Animal has virtual methods
  // Use when you know the runtime type will always be exactly as declared
  fory::field<std::shared_ptr<Animal>, 1, fory::nullable, fory::dynamic<false>> fixed_animal;
};
FORY_STRUCT(Zoo, animal, fixed_animal);

Valid for: std::shared_ptr<T>, std::unique_ptr<T>

Combining Tags

Multiple tags can be combined for shared pointers:

// Nullable + ref tracking
fory::field<std::shared_ptr<Node>, 0, fory::nullable, fory::ref> link;

Type Rules

Type	Allowed Options	Nullability
Primitives, strings	None	Use `std::optional<T>` if nullable
`std::optional<T>`	None	Inherently nullable
`std::shared_ptr<T>`	`nullable`, `ref`, `dynamic<V>`	Non-null by default
`std::unique_ptr<T>`	`nullable`, `dynamic<V>`	Non-null by default

Complete Example

#include "fory/serialization/fory.h"

using namespace fory::serialization;

// Define a struct with various field configurations
struct Document {
  // Required fields (non-nullable)
  fory::field<std::string, 0> title;
  fory::field<int32_t, 1> version;

  // Optional primitive using std::optional
  fory::field<std::optional<std::string>, 2> description;

  // Nullable pointer
  fory::field<std::unique_ptr<std::string>, 3, fory::nullable> metadata;

  // Reference-tracked shared pointer
  fory::field<std::shared_ptr<Document>, 4, fory::ref> parent;

  // Nullable + reference-tracked
  fory::field<std::shared_ptr<Document>, 5, fory::nullable, fory::ref> related;
};
FORY_STRUCT(Document, title, version, description, metadata, parent, related);

int main() {
  auto fory = Fory::builder().xlang(true).build();
  fory.register_struct<Document>(100);

  Document doc;
  doc.title = "My Document";
  doc.version = 1;
  doc.description = "A sample document";
  doc.metadata = nullptr;  // Allowed because nullable
  doc.parent = std::make_shared<Document>();
  doc.parent->title = "Parent Doc";
  doc.related = nullptr;  // Allowed because nullable

  auto bytes = fory.serialize(doc).value();
  auto decoded = fory.deserialize<Document>(bytes).value();
}

Compile-Time Validation

Invalid configurations are caught at compile time:

// Error: nullable and not_null are mutually exclusive
fory::field<std::shared_ptr<int>, 0, fory::nullable, fory::not_null> bad1;

// Error: nullable only valid for smart pointers
fory::field<int32_t, 0, fory::nullable> bad2;

// Error: ref only valid for shared_ptr
fory::field<std::unique_ptr<int>, 0, fory::ref> bad3;

// Error: options not allowed for std::optional (inherently nullable)
fory::field<std::optional<int>, 0, fory::nullable> bad4;

Backwards Compatibility

Existing structs without fory::field<> wrappers continue to work:

// Old style - still works
struct LegacyPerson {
  std::string name;
  int32_t age;
};
FORY_STRUCT(LegacyPerson, name, age);

// New style with field metadata
struct ModernPerson {
  fory::field<std::string, 0> name;
  fory::field<int32_t, 1> age;
};
FORY_STRUCT(ModernPerson, name, age);

FORY_FIELD_TAGS Macro

The FORY_FIELD_TAGS macro provides a non-invasive way to add field metadata without modifying struct definitions. This is useful for:

Third-party types: Add metadata to types you don't own
Clean structs: Keep struct definitions as pure C++
Isolated dependencies: Confine Fory headers to serialization config files

Usage

// user_types.h - NO fory headers needed!
struct Document {
  std::string title;
  int32_t version;
  std::optional<std::string> description;
  std::shared_ptr<User> author;
  std::shared_ptr<User> reviewer;
  std::shared_ptr<Document> parent;
  std::unique_ptr<Data> data;
};

// serialization_config.cpp - fory config isolated here
#include "fory/serialization/fory.h"
#include "user_types.h"

FORY_STRUCT(Document, title, version, description, author, reviewer, parent, data)

FORY_FIELD_TAGS(Document,
  (title, 0),                      // string: non-nullable
  (version, 1),                    // int: non-nullable
  (description, 2),                // optional: inherently nullable
  (author, 3),                     // shared_ptr: non-nullable (default)
  (reviewer, 4, nullable),         // shared_ptr: nullable
  (parent, 5, ref),                // shared_ptr: non-nullable, with ref tracking
  (data, 6, nullable)              // unique_ptr: nullable
)

FORY_FIELD_TAGS Options

Field Type	Valid Combinations
Primitives, strings	`(field, id)` only
`std::optional<T>`	`(field, id)` only
`std::shared_ptr<T>`	`(field, id)`, `(field, id, nullable)`, `(field, id, ref)`, `(field, id, nullable, ref)`
`std::unique_ptr<T>`	`(field, id)`, `(field, id, nullable)`

API Comparison

Aspect	`fory::field<>` Wrapper	`FORY_FIELD_TAGS` Macro
Struct definition	Modified (wrapped types)	Unchanged (pure C++)
IDE support	Template noise	Excellent (clean types)
Third-party classes	Not supported	Supported
Header dependencies	Required everywhere	Isolated to config
Migration effort	High (change all fields)	Low (add one macro)

FORY_FIELD_CONFIG Macro

The FORY_FIELD_CONFIG macro is the most powerful and flexible way to configure field-level serialization. It provides:

Builder pattern API: Fluent, chainable configuration with F(id).option1().option2()
Encoding control: Specify how unsigned integers are encoded (varint, fixed, tagged)
Compile-time verification: Field names are verified against member pointers
Cross-language compatibility: Configure encoding to match other languages (Java, Rust, etc.)

Basic Syntax

FORY_FIELD_CONFIG(StructType,
    (field1, fory::F(0)),                           // Simple: just ID
    (field2, fory::F(1).nullable()),                // With nullable
    (field3, fory::F(2).varint()),                  // With encoding
    (field4, fory::F(3).nullable().ref()),          // Multiple options
    (field5, 4)                                     // Backward compatible: integer ID
);

The F() Builder

The fory::F(id) factory creates a FieldMeta object that supports method chaining:

fory::F(0)                    // Create with field ID 0
    .nullable()               // Mark as nullable
    .ref()                    // Enable reference tracking
    .varint()                 // Use variable-length encoding
    .fixed()                  // Use fixed-size encoding
    .tagged()                 // Use tagged encoding
    .dynamic(false)           // Skip type info (no dynamic dispatch)
    .dynamic(true)            // Force type info (enable dynamic dispatch)
    .compress(false)          // Disable compression

Tip: To use F() without the fory:: prefix, add a using declaration:

using fory::F;

FORY_FIELD_CONFIG(MyStruct,
    (field1, F(0).varint()),      // No prefix needed
    (field2, F(1).nullable())
);

Encoding Options for Unsigned Integers

For uint32_t and uint64_t fields, you can specify the wire encoding:

Method	Type ID	Description	Use Case
`.varint()`	VAR_UINT32/64	Variable-length encoding (1-5 or 1-10 bytes)	Values typically small
`.fixed()`	UINT32/64	Fixed-size encoding (always 4 or 8 bytes)	Values uniformly distributed
`.tagged()`	TAGGED_UINT64	Tagged hybrid encoding with size hint (uint64)	Mixed small and large values (uint64)

Note: uint8_t and uint16_t always use fixed encoding (UINT8, UINT16).

Complete Example

#include "fory/serialization/fory.h"

using namespace fory::serialization;

// Define struct with unsigned integer fields
struct MetricsData {
  // Counters - often small values, use varint for space efficiency
  uint32_t requestCount;
  uint64_t bytesSent;

  // IDs - uniformly distributed, use fixed for consistent performance
  uint32_t userId;
  uint64_t sessionId;

  // Timestamps - use tagged encoding for mixed value ranges
  uint64_t createdAt;

  // Nullable fields
  std::optional<uint32_t> errorCount;
  std::optional<uint64_t> lastAccessTime;
};

FORY_STRUCT(MetricsData, requestCount, bytesSent, userId, sessionId,
            createdAt, errorCount, lastAccessTime);

// Configure field encoding
FORY_FIELD_CONFIG(MetricsData,
    // Small counters - varint saves space
    (requestCount, fory::F(0).varint()),
    (bytesSent, fory::F(1).varint()),

    // IDs - fixed for consistent performance
    (userId, fory::F(2).fixed()),
    (sessionId, fory::F(3).fixed()),

    // Timestamp - tagged encoding
    (createdAt, fory::F(4).tagged()),

    // Nullable fields
    (errorCount, fory::F(5).nullable().varint()),
    (lastAccessTime, fory::F(6).nullable().tagged())
);

int main() {
  auto fory = Fory::builder().xlang(true).build();
  fory.register_struct<MetricsData>(100);

  MetricsData data{
      .requestCount = 42,
      .bytesSent = 1024,
      .userId = 12345678,
      .sessionId = 9876543210,
      .createdAt = 1704067200000000000ULL, // 2024-01-01 in nanoseconds
      .errorCount = 3,
      .lastAccessTime = std::nullopt
  };

  auto bytes = fory.serialize(data).value();
  auto decoded = fory.deserialize<MetricsData>(bytes).value();
}

Cross-Language Compatibility

When serializing data to be read by other languages, use FORY_FIELD_CONFIG to match their encoding expectations:

Java Compatibility:

// Java uses these type IDs for unsigned integers:
// - Byte (u8): UINT8 (fixed)
// - Short (u16): UINT16 (fixed)
// - Integer (u32): VAR_UINT32 (varint) or UINT32 (fixed)
// - Long (u64): VAR_UINT64 (varint), UINT64 (fixed), or TAGGED_UINT64

struct JavaCompatible {
  uint8_t byteField;      // Maps to Java Byte
  uint16_t shortField;    // Maps to Java Short
  uint32_t intVarField;   // Maps to Java Integer with varint
  uint32_t intFixedField; // Maps to Java Integer with fixed
  uint64_t longVarField;  // Maps to Java Long with varint
  uint64_t longTagged;    // Maps to Java Long with tagged
};

FORY_STRUCT(JavaCompatible, byteField, shortField, intVarField,
            intFixedField, longVarField, longTagged);

FORY_FIELD_CONFIG(JavaCompatible,
    (byteField, fory::F(0)),                    // UINT8 (auto)
    (shortField, fory::F(1)),                   // UINT16 (auto)
    (intVarField, fory::F(2).varint()),         // VAR_UINT32
    (intFixedField, fory::F(3).fixed()),        // UINT32
    (longVarField, fory::F(4).varint()),        // VAR_UINT64
    (longTagged, fory::F(5).tagged())           // TAGGED_UINT64
);

Schema Evolution with FORY_FIELD_CONFIG

In compatible mode, fields can have different nullability between sender and receiver:

// Version 1: All fields non-nullable
struct DataV1 {
  uint32_t id;
  uint64_t timestamp;
};
FORY_STRUCT(DataV1, id, timestamp);
FORY_FIELD_CONFIG(DataV1,
    (id, fory::F(0).varint()),
    (timestamp, fory::F(1).tagged())
);

// Version 2: Added nullable fields
struct DataV2 {
  uint32_t id;
  uint64_t timestamp;
  std::optional<uint32_t> version;  // New nullable field
};
FORY_STRUCT(DataV2, id, timestamp, version);
FORY_FIELD_CONFIG(DataV2,
    (id, fory::F(0).varint()),
    (timestamp, fory::F(1).tagged()),
    (version, fory::F(2).nullable().varint())  // New field with nullable
);

FORY_FIELD_CONFIG Options Reference

Method	Description	Valid For
`.nullable()`	Mark field as nullable	Smart pointers, primitives
`.ref()`	Enable reference tracking	`std::shared_ptr` only
`.dynamic(true)`	Force type info to be written (dynamic dispatch)	Smart pointers
`.dynamic(false)`	Skip type info (use declared type directly)	Smart pointers
`.varint()`	Use variable-length encoding	`uint32_t`, `uint64_t`
`.fixed()`	Use fixed-size encoding	`uint32_t`, `uint64_t`
`.tagged()`	Use tagged hybrid encoding	`uint64_t` only
`.compress(v)`	Enable/disable field compression	All types

Comparing Field Configuration Macros

Feature	`fory::field<>`	`FORY_FIELD_TAGS`	`FORY_FIELD_CONFIG`
Struct modification	Required (wrap types)	None	None
Encoding control	No	No	Yes (varint/fixed/tagged)
Builder pattern	No	No	Yes
Compile-time verify	Yes	Limited	Yes (member pointers)
Cross-lang compat	Limited	Limited	Full
Recommended for	Simple structs	Third-party types	Complex/xlang structs

Default Values

Nullable: Only std::optional<T> is nullable by default; all other types (including std::shared_ptr) are non-nullable
Ref tracking: Disabled by default for all types (including std::shared_ptr)

You need to configure fields when:

A field can be null (use std::optional<T> or mark with nullable())
A field needs reference tracking for shared/circular objects (use ref())
Integer types need specific encoding for cross-language compatibility
You want to reduce metadata size (use field IDs)

// Xlang mode: explicit configuration required
struct User {
    std::string name;                              // Non-nullable by default
    std::optional<std::string> email;              // Nullable (std::optional)
    std::shared_ptr<User> friend_ptr;              // Ref tracking by default
};

FORY_STRUCT(User, name, email, friend_ptr);

FORY_FIELD_CONFIG(User,
    (name, fory::F(0)),
    (email, fory::F(1)),                           // nullable implicit for optional
    (friend_ptr, fory::F(2).nullable().ref())      // explicit nullable + ref
);

Default Values Summary

Type	Default Nullable	Default Ref Tracking
Primitives, `string`	`false`	`false`
`std::optional<T>`	`true`	`false`
`std::shared_ptr<T>`	`false`	`true`
`std::unique_ptr<T>`	`false`	`false`

Type Registration - Registering types with FORY_STRUCT
Schema Evolution - Using tag IDs for schema evolution
Configuration - Enabling reference tracking globally
Cross-Language - Interoperability with Java, Rust, Python

Overview​

The fory::field Template​

Template Parameters​

Basic Usage​

Tag Types​

fory::nullable​

fory::not_null​

fory::ref​

fory::dynamic<V>​

Combining Tags​

Type Rules​

Complete Example​

Compile-Time Validation​

Backwards Compatibility​

FORY_FIELD_TAGS Macro​

Usage​

FORY_FIELD_TAGS Options​

API Comparison​

FORY_FIELD_CONFIG Macro​

Basic Syntax​

The F() Builder​

Encoding Options for Unsigned Integers​

Complete Example​

Cross-Language Compatibility​

Schema Evolution with FORY_FIELD_CONFIG​

FORY_FIELD_CONFIG Options Reference​

Comparing Field Configuration Macros​

Default Values​

Default Values Summary​

Related Topics​

Overview

The fory::field Template

Template Parameters

Basic Usage

Tag Types

fory::nullable

fory::not_null

fory::ref

fory::dynamic<V>

Combining Tags

Type Rules

Complete Example

Compile-Time Validation

Backwards Compatibility

FORY_FIELD_TAGS Macro

Usage

FORY_FIELD_TAGS Options

API Comparison

FORY_FIELD_CONFIG Macro

Basic Syntax

The F() Builder

Encoding Options for Unsigned Integers

Complete Example

Cross-Language Compatibility

Schema Evolution with FORY_FIELD_CONFIG

FORY_FIELD_CONFIG Options Reference

Comparing Field Configuration Macros

Default Values

Default Values Summary

Related Topics