Customization

Many parts of SymbolicRegression.jl are designed to be customizable.

The normal way to do this in Julia is to define a new type that subtypes an abstract type from a package, and then define new methods for the type, extending internal methods on that type.

Custom Options

For example, you can define a custom options type:

SymbolicRegression.CoreModule.OptionsStructModule.AbstractOptions Type

AbstractOptions

An abstract type that stores all search hyperparameters for SymbolicRegression.jl. The standard implementation is Options.

You may wish to create a new subtypes of AbstractOptions to override certain functions or create new behavior. Ensure that this new type has all properties of Options.

For example, if we have new options that we want to add to Options:

Base.@kwdef struct MyNewOptions
    a::Float64 = 1.0
    b::Int = 3
end

we can create a combined options type that forwards properties to each corresponding type:

struct MyOptions{O<:SymbolicRegression.Options} <: SymbolicRegression.AbstractOptions
    new_options::MyNewOptions
    sr_options::O
end
const NEW_OPTIONS_KEYS = fieldnames(MyNewOptions)

# Constructor with both sets of parameters:
function MyOptions(; kws...)
    new_options_keys = filter(k -> k in NEW_OPTIONS_KEYS, keys(kws))
    new_options = MyNewOptions(; NamedTuple(new_options_keys .=> Tuple(kws[k] for k in new_options_keys))...)
    sr_options_keys = filter(k -> !(k in NEW_OPTIONS_KEYS), keys(kws))
    sr_options = SymbolicRegression.Options(; NamedTuple(sr_options_keys .=> Tuple(kws[k] for k in sr_options_keys))...)
    return MyOptions(new_options, sr_options)
end

# Make all `Options` available while also making `new_options` accessible
function Base.getproperty(options::MyOptions, k::Symbol)
    if k in NEW_OPTIONS_KEYS
        return getproperty(getfield(options, :new_options), k)
    else
        return getproperty(getfield(options, :sr_options), k)
    end
end

Base.propertynames(options::MyOptions) = (NEW_OPTIONS_KEYS..., fieldnames(SymbolicRegression.Options)...)

which would let you access a and b from MyOptions objects, as well as making all properties of Options available for internal methods in SymbolicRegression.jl

source

Any function in SymbolicRegression.jl you can generally define a new method on your custom options type, to define custom behavior.

Custom Mutations

Define a custom mutation by subtyping AbstractMutation, implementing mutate!, and passing it with a weight through Options(; mutations=...).

Here is a mutation that replaces a random subtree with a single variable:

using SymbolicRegression
using SymbolicRegression: AbstractMutation, MutationResult
using DynamicExpressions: get_contents, with_contents, AbstractExpression, AbstractExpressionNode

struct PruneMutation <: AbstractMutation end

function SymbolicRegression.mutate!(
    new_tree::N, parent_member::P, ::PruneMutation, options; nfeatures, kws...
) where {N<:AbstractExpression,P}
    tree = get_contents(new_tree)
    # Find a random non-leaf node and replace it with a variable
    nodes = filter(n -> n.degree > 0, collect(tree))
    if !isempty(nodes)
        target = rand(nodes)
        target.degree = 0
        target.feature = rand(1:nfeatures)
    end
    return MutationResult{N,P}(; tree=new_tree)
end

Pass it to Options with a weight. New mutation types are added alongside the defaults; to replace or remove a default, pass default_mutations=():

model = SRRegressor(
    binary_operators=[+, -, *, /],
    unary_operators=[cos],
    mutations=[PruneMutation() => 0.1],
)

SymbolicRegression.MutateModule.mutate! Function

mutate!(
    new_tree::N,
    parent_member::P,
    mutation::AbstractMutation,
    options::AbstractOptions;
    kws...,
) where {N<:AbstractExpression,P<:AbstractPopMember}

Perform mutation on the offspring new_tree (a fresh scratch copy of the parent's tree). parent_member carries parent metadata (cost, loss, ref, etc.) that some mutations need.

Add a new mutation by defining a struct subtyping AbstractMutation and a matching mutate! method.

Keywords

• dataset::Dataset: The dataset used for scoring.

• cost: The cost of parent_member before mutation.

• loss: The loss of parent_member before mutation.

• curmaxsize: The current maximum size constraint, which may differ from options.maxsize.

• nfeatures: The number of features in the dataset.

• parent_ref: Reference to parent_member's parent (used for lineage logging).

• attempt::Int: 1-based attempt number within the engine's constraint-retry loop. Expensive mutations can use this to behave differently on retries.

• trace::MaybeTrace: Mutation tracing state, or nothing when tracing is disabled.

• context: per-call mutable context for the selected mutation type (built by prepare_mutation_context and conditioned by plugins via condition_mutation!); nothing for mutations without one.

• plugin_states::Tuple: The active worker plugin states, in tuple order matching options.plugins.

Returns

A MutationResult{N,P} object containing the mutated tree or member (but not both), the number of evaluations performed, if any, and whether to return immediately from the mutation function, or to let the next_generation function handle accepting or rejecting the mutation. For example, a simplify operation will not change the loss, so it can always return immediately.

source

SymbolicRegression.CoreModule.MutationsModule.AbstractMutation Type

AbstractMutation

A mutation kind is a struct (often Base.@kwdef for per-mutation config) subtyping AbstractMutation. The engine dispatches the per-cycle mutate! method on the mutation's type; weight sampling, plugin observation hooks, and condition_mutation_weights! all key off the type.

To add a new mutation kind, define a struct + a mutate! method:

struct MyMutation <: AbstractMutation end

function SymbolicRegression.mutate!(
    new_tree, parent_member, ::MyMutation, options; kws...
)
    # ... modify new_tree ...
    return SymbolicRegression.MutationResult{
        typeof(new_tree),typeof(parent_member)
    }(; tree=new_tree)
end

Then include it in Options(; mutations = [MyMutation() => 0.1]). An explicit mutation replaces a default of the same type; new mutation types are added. Pass default_mutations=() to disable every automatic default.

Experimental

source

SymbolicRegression.MutateModule.condition_mutation_weights! Function

condition_mutation_weights!(weights, member::AbstractPopMember, options, curmaxsize, nfeatures)

Adjust the mutation weights (a Vector{Pair{AbstractMutation,Float64}}) based on the properties of the current member and options — e.g. disable operator-mutation when the tree has no operators, disable simplify when options.should_simplify is false, etc.

Plugin overloads should use _set_weight!(weights, MyMutation, w) to modify the per-mutation weight in place.

source

condition_mutation_weights!(weights, state, plugin, member, options, curmaxsize, nfeatures)

Plugin-dispatched method: called once per plugin in tuple order after the engine's legality conditioning. Default is a no-op.

Experimental

source

SymbolicRegression.CoreModule.MutationWeightsModule.sample_mutation Function

sample_mutation(mutations) -> AbstractMutation

Pick a mutation kind by weight. Returns the singleton instance.

Marked @unstable because the return type is AbstractMutation — the concrete subtype is selected at runtime by weighted sampling. The caller hands the result to mutate!, which dispatches per concrete type, so the instability is contained.

source

SymbolicRegression.MutateModule.MutationResult Type

MutationResult{N<:AbstractExpression,P<:AbstractPopMember}

Represents the result of a mutation operation in the genetic programming algorithm. This struct is used to return values from mutate! functions.

Fields

• tree::Union{N, Nothing}: The mutated expression tree, if applicable. Either tree or member must be set, but not both.

• member::Union{P, Nothing}: The mutated population member, if applicable. Either member or tree must be set, but not both.

• num_evals::Float64: The number of evaluations performed during the mutation, which is automatically set to 0.0. Only used for things like optimize.

• return_immediately::Bool: If true, the mutation process should return immediately, bypassing further checks, used for things like simplify or optimize where you already know the loss value of the result.

Usage

This struct encapsulates the result of a mutation operation. Either a new expression tree or a new population member is returned, but not both.

Return the member if you want to return immediately, and have computed the loss value as part of the mutation.

source

Custom Crossovers

Define a custom crossover by subtyping AbstractCrossover, implementing crossover, and passing it with a weight through Options(; crossovers=...). Whenever the engine selects crossover (via crossover_probability), it samples one crossover kind by weight from options.crossovers and retries it on constraint failures, up to an attempt limit.

Here is a crossover that, instead of swapping subtrees, combines both parents wholesale under a random binary operator (so x + y and cos(x) might produce (x + y) * cos(x)):

using SymbolicRegression
using SymbolicRegression: AbstractCrossover, CrossoverResult
using DynamicExpressions: get_contents, with_contents

struct RootCrossover <: AbstractCrossover end

function SymbolicRegression.crossover(
    member1::P, member2::P, ::RootCrossover, options; kws...
) where {T,L,N,P<:PopMember{T,L,N}}
    t1 = get_contents(member1.tree)
    t2 = get_contents(member2.tree)
    op1, op2 = rand(1:length(options.operators.binops), 2)
    child1 = with_contents(member1.tree, Node(; op=op1, l=copy(t1), r=copy(t2)))
    child2 = with_contents(member2.tree, Node(; op=op2, l=copy(t2), r=copy(t1)))
    return CrossoverResult{N}(; child1, child2)
end

Pass it to Options with a weight. New crossover types are added alongside the default SubtreeCrossover; to remove the default, pass default_crossovers=():

model = SRRegressor(
    binary_operators=[+, -, *, /],
    unary_operators=[cos],
    crossovers=[RootCrossover() => 0.2],
)

The engine retries the sampled crossover when the children violate constraints, passing a 1-based attempt keyword each time. A crossover that is expensive to run (e.g. one backed by an external model) can check attempt and return copies of the parents' trees on retries instead of re-running.

Missing docstring.

Missing docstring for crossover. Check Documenter's build log for details.

SymbolicRegression.CoreModule.CrossoversModule.AbstractCrossover Type

AbstractCrossover

A crossover kind is a struct (often Base.@kwdef for per-crossover config) subtyping AbstractCrossover. The engine dispatches the per-event crossover method on the crossover's type; weight sampling keys off the type.

To add a new crossover kind, define a struct + a crossover method:

struct MyCrossover <: AbstractCrossover end

function SymbolicRegression.crossover(
    member1, member2, ::MyCrossover, options; kws...
)
    child1, child2 = ...  # combine the parents' trees
    return SymbolicRegression.CrossoverResult{typeof(child1)}(; child1, child2)
end

Then include it in Options(; crossovers = [MyCrossover() => 0.1]). An explicit crossover replaces a default of the same type; new crossover types are added. Pass default_crossovers=() to disable every automatic default.

Experimental

source

Missing docstring.

Missing docstring for CrossoverResult. Check Documenter's build log for details.

Custom Expressions

You can create your own expression types by defining a new type that extends AbstractExpression.

DynamicExpressions.ExpressionModule.AbstractExpression Type

AbstractExpression{T,N}

(Experimental) Abstract type for user-facing expression types, which contain both the raw expression tree operating on a value type of T, as well as associated metadata to evaluate and render the expression.

See ExpressionInterface for a full description of the interface implementation, as well as tests to verify correctness.

If you wish to use @parse_expression, you can also customize the parsing behavior with

• parse_leaf

source

The interface is fairly flexible, and permits you define specific functional forms, extra parameters, etc. See the documentation of DynamicExpressions.jl for more details on what methods you need to implement. You can test the implementation of a given interface by using ExpressionInterface which makes use of Interfaces.jl:

DynamicExpressions.InterfacesModule.ExpressionInterface Type

    ExpressionInterface

An Interfaces.jl Interface with mandatory components (:get_contents, :get_metadata, :get_tree, :get_operators, :get_variable_names, :copy, :with_contents, :with_metadata) and optional components (:copy_into!, :count_nodes, :count_constant_nodes, :count_depth, :index_constant_nodes, :has_operators, :has_constants, :get_scalar_constants, :set_scalar_constants!, :string_tree, :default_node_type, :constructorof, :tree_mapreduce).

Defines the interface of AbstractExpression for user-facing expression types, which can store operators, extra parameters, functional forms, variable names, etc.

Extended help

Mandatory keys:

• get_contents: extracts the runtime contents of an expression

• get_metadata: extracts the runtime metadata of an expression

• get_tree: extracts the expression tree from AbstractExpression

• get_operators: returns the operators used in the expression (or pass operators explicitly to override)

• get_variable_names: returns the variable names used in the expression (or pass variable_names explicitly to override)

• copy: returns a copy of the expression

• with_contents: returns the expression with different tree

• with_metadata: returns the expression with different metadata

Optional keys:

• copy_into!: copies an expression into a preallocated container

• count_nodes: counts the number of nodes in the expression tree

• count_constant_nodes: counts the number of constant nodes in the expression tree

• count_depth: calculates the depth of the expression tree

• index_constant_nodes: indexes constants in the expression tree

• has_operators: checks if the expression has operators

• has_constants: checks if the expression has constants

• get_scalar_constants: gets constants from the expression tree, returning a tuple of: (1) a flat vector of the constants, and (2) an reference object that can be used by set_scalar_constants! to efficiently set them back

• set_scalar_constants!: sets constants in the expression tree, given: (1) a flat vector of constants, (2) the expression, and (3) the reference object produced by get_scalar_constants

• string_tree: returns a string representation of the expression tree

• default_node_type: returns the default node type for the expression

• constructorof: gets the constructor function for a type

• tree_mapreduce: applies a function across the tree

source

Then, for SymbolicRegression.jl, you would pass expression_type to the Options constructor, as well as any expression_options you need (as a NamedTuple).

If needed, you may need to overload SymbolicRegression.ExpressionBuilder.extra_init_params in case your expression needs additional parameters. See src/TemplateExpression.jl for an example.

You can also look at src/TemplateExpression.jl for a custom expression type used by SymbolicRegression.jl.

Plugins

See the Plugins page for how to hook into the search loop with custom lifecycle callbacks, selection biases, and population seeding.

Other Customizations

Other internal abstract types include the following:

SymbolicRegression.SearchUtilsModule.AbstractRuntimeOptions Type

AbstractRuntimeOptions

An abstract type representing runtime configuration parameters for the symbolic regression algorithm.

AbstractRuntimeOptions is used by equation_search to control runtime aspects such as parallelism and iteration limits. By subtyping AbstractRuntimeOptions, advanced users can customize runtime behaviors by passing it to equation_search.

See Also

• RuntimeOptions: Default implementation used by equation_search.

• equation_search: Main function to perform symbolic regression.

• AbstractOptions: See how to extend abstract types for customizing options.

source

SymbolicRegression.SearchUtilsModule.AbstractSearchState Type

AbstractSearchState{T,L,N}

An abstract type encapsulating the internal state of the search process during symbolic regression.

AbstractSearchState instances hold information like populations and progress metrics, used internally by equation_search. Subtyping AbstractSearchState allows customization of search state management.

Look through the source of equation_search to see how this is used.

See Also

• SearchState: Default implementation of AbstractSearchState.

• equation_search: Function where AbstractSearchState is utilized.

• AbstractOptions: See how to extend abstract types for customizing options.

source

These let you include custom state variables and runtime options.