I/O API

File input/output for simulation data.

NetCDF Output

NetCDFFileHandler

Tarang.NetCDFFileHandler — Type

NetCDF File Handler matching Tarang H5FileHandler structure

Follows Tarang pattern:

basepath/handlernames1/handlernames1p0.nc for processor files
basepath/handlername_s1.nc for gathered files
/scales/ group with time coordinates
/tasks/ group with field data

source

addnetcdfhandler

Tarang.add_netcdf_handler — Function

Convenience function to create NetCDF file handler (matching Tarang API) Usage: handler = addfilehandler("snapshots", dist, vars, simdt=0.25, maxwrites=50)

source

Create a file handler for NetCDF output.

addfilehandler

Tarang.add_file_handler — Function

Tarang-style helper to create a NetCDF file handler. Matches evaluator.addfilehandler(...) usage in Tarang.

source

Add file handler for output

source

Add file handler with automatic format detection

Supported formats:

:netcdf or :nc - NetCDF format
:auto - Auto-detect from file extension (defaults to NetCDF)

source

Alternative constructor for file handlers.

handler = add_netcdf_handler(
    base_path,          # Output path/name
    dist,               # Distributor
    fields_dict;        # Dict of fields
    parallel="gather",  # I/O mode
    max_writes=100      # Files before rollover
)

Arguments:

base_path: Base path for output files
dist: MPI distributor
fields_dict: Dictionary mapping names to fields
parallel: "gather" (single file) or "virtual" (per-process)
max_writes: Maximum writes per file before creating new file

Returns: NetCDFFileHandler

add_task!

Tarang.add_task! — Function

Add task to handler (matching Tarang API)

Example usage (matching Tarang style):

snapshots = evaluator.addfilehandler('snapshots', simdt=0.25, maxwrites=50) snapshots.addtask(b, name='buoyancy') snapshots.addtask(-d3.div(d3.skew(u)), name='vorticity')

source

Add field or operator to file handler

source

Add field or operator to file handler with explicit name

source

Add a field/operator task to the dictionary handler.

source

Add a field/array to be written by this file handler.

source

Add a field output task.

add_task

Tarang.add_task — Function

Alias without bang for Tarang-style task addition.

source

Add a field output task (alternative syntax).

add_task!(handler, field; name="field_name")

With postprocessing:

add_task!(handler, field;
    name="processed",
    postprocess=data -> mean(data, dims=1)
)

addprofiletask!

Tarang.add_profile_task! — Function

Add a task that computes a 1D profile (mean over all but one dimension).

Arguments

handler: NetCDFFileHandler instance
field: Field to compute profile of
dim: The dimension to keep (profile along this dimension)
name: Optional name for the output variable

Example

add_profile_task!(handler, u, dim=:z, name="u_profile_z")

source

Add a task that computes mean profile over specified dimensions.

# Mean over x (dimension 1) - produces z-profile
add_profile_task!(handler, field; dims=1, name="field_profile")

# Mean over x and y
add_profile_task!(handler, field; dims=(1,2), name="field_profile_xy")

addmeantask!

Tarang.add_mean_task! — Function

Add a task that computes the mean over specified dimensions.

Arguments

handler: NetCDFFileHandler instance
field: Field to compute mean of
dims: Dimensions to average over (e.g., (:x, :y) or (1, 2))
name: Optional name for the output variable

Example

add_mean_task!(handler, u, dims=(:x, :y), name="u_mean_z")

source

Add a task that computes mean values.

add_mean_task!(handler, field; name="field_mean")

addslicetask!

Tarang.add_slice_task! — Function

Add a task that extracts a slice of a field.

Arguments

handler: NetCDFFileHandler instance
field: Field to slice
slices: Dictionary or named tuple specifying slice positions e.g., Dict(:z => 0.5) or (z=0.5,) for midplane slice
name: Optional name for the output variable

Example

add_slice_task!(handler, u, slices=Dict(:z => 0.0), name="u_bottom")
add_slice_task!(handler, T, slices=(x=0.5, y=0.5), name="T_centerline")

source

Add a task that extracts a slice.

# Slice at index
add_slice_task!(handler, field; dim=1, idx=64, name="field_slice")

# Using slices dictionary
add_slice_task!(handler, field; slices=Dict(1 => 32), name="slice")

addrmstask!

Tarang.add_rms_task! — Function

Add a task that computes the RMS (root mean square) over specified dimensions.

source

Add a task that computes RMS (root-mean-square) values.

add_rms_task!(handler, field; name="field_rms")

addvariancetask!

Tarang.add_variance_task! — Function

Add a task that computes the variance over specified dimensions.

source

Add a task that computes variance.

add_variance_task!(handler, field; name="field_variance")

addextrematask!

Tarang.add_extrema_task! — Function

Add a task that computes min/max over the domain.

source

Add a task that tracks minimum and maximum values.

add_extrema_task!(handler, field; name="field_extrema")

process!

Tarang.process! — Function

Process handler: write all tasks to NetCDF (matching Tarang process method)

source

process!(handler::VirtualFileHandler, solver, wall_time, sim_time, iteration)

Write per-rank data files and rank-0 manifest file.

source

Write pending data to file.

process!(handler;
    iteration=solver.iteration,
    wall_time=elapsed,
    sim_time=solver.sim_time,
    timestep=dt
)

Handler Management

check_schedule

Tarang.check_schedule — Function

Check if handler should process based on schedule (matching Tarang logic)

Always writes initial conditions (iteration=0) when any scheduling mode is configured. For subsequent iterations, uses the configured cadence.

source

Check if the handler should write based on schedule.

createcurrentfile!

Tarang.create_current_file! — Function

Create NetCDF file with Tarang-style structure

source

Create a new output file.

current_path

Tarang.current_path — Function

Get current set path following Tarang naming: handlernames1/

source

Get the current output file path.

filepath = current_path(handler)

current_file

Tarang.current_file — Function

Get current file path following Tarang naming: handlernames1/handlernames1_p0.nc

source

Get the current output file (alternative to current_path).

filepath = current_file(handler)

getoutputfiles

Tarang.get_output_files — Function

Get list of all output files created by this handler.

source

Get list of all output files created by handler.

gethandlerinfo

Tarang.get_handler_info — Function

Get metadata about the handler's output.

source

Get information about the handler state.

close!

Tarang.close! — Function

Close handler and finalize all files.

Writes final metadata attributes to the current NetCDF file. Also clears the internal variable-creation cache.

Note: NetCDF.jl's filename-based API (nccreate, ncwrite, ncread, ncputatt) opens and closes the underlying file descriptor on each call, so there is no persistent file handle to release here. This method exists to flush final metadata and reset handler state.

source

Close the handler and finalize output.

close!(handler)

reset!

Tarang.reset! — Function

Reset handler for a new simulation run.

source

reset!(filter::ExponentialMean)

Reset the filter state to zero.

source

reset!(filter::ButterworthFilter)

Reset the filter state to zero.

source

reset!(filter::LagrangianFilter)

Reset the Lagrangian filter state.

source

reset!(model::EddyViscosityModel)

Reset the eddy viscosity field to zero. GPU-aware: fill!() works for both CPU and GPU arrays.

source

Reset the handler state.

File Merging

NetCDFMerger

Tarang.NetCDFMerger — Type

NetCDF File Merger - handles merging of per-processor files

source

mergenetcdffiles

Tarang.merge_netcdf_files — Function

merge_netcdf_files(base_name; kwargs...)

Merge per-processor NetCDF files into a single merged file.

Arguments

base_name::String: Base name of the handler (e.g., "snapshots", "analysis")
set_number::Int=1: Set number to merge (default: 1)
output_name::String="": Output filename (default: auto-generated)
merge_mode::MergeMode=RECONSTRUCT: How to combine processor data
cleanup::Bool=false: Delete source files after successful merge
verbose::Bool=true: Print progress information

Examples

# Basic merge
merge_netcdf_files("snapshots")

# Advanced options  
merge_netcdf_files("analysis", 
                   set_number=2,
                   output_name="analysis_complete.nc", 
                   cleanup=true,
                   merge_mode=SIMPLE_CONCAT)

source

Merge multiple NetCDF files from parallel output.

merge_netcdf_files(base_path; output_file="merged.nc")

merge_files!

Tarang.merge_files! — Function

Main merge function - orchestrates the entire merging process

source

batchmergenetcdf

Tarang.batch_merge_netcdf — Function

batch_merge_netcdf(handlers; kwargs...)

Merge multiple handlers in batch mode.

Examples

# Merge multiple handlers
batch_merge_netcdf(["snapshots", "analysis", "checkpoints"])

# With cleanup
batch_merge_netcdf(["snapshots", "analysis"], cleanup=true)

source

Merge multiple sets of parallel output files.

batch_merge_netcdf(["output1", "output2", "output3"])

findmergeablehandlers

Tarang.find_mergeable_handlers — Function

find_mergeable_handlers(directory=".")

Find all handlers with processor files ready for merging.

Returns a dictionary mapping handler names to available set numbers.

source

Find handlers that can be merged.

cleanupsourcefiles!

Tarang.cleanup_source_files! — Function

Clean up source processor files after successful merge

source

Remove source files after merging.

Equation Parsing

Internal functions for parsing equation strings.

split_equation

Tarang.split_equation — Function

split_equation(equation::AbstractString)

Split an equation string into left- and right-hand sides, tracking bracket depth so that keyword arguments (e.g. $f(x=1)$) and array indices do not trigger false splits.

Tracks parentheses (), square brackets [], and curly braces {}.

Throws SymbolicParsingError if there is not exactly one top-level equals sign.

source

split_call

Tarang.split_call — Function

split_call(call::AbstractString)

Split a function-style string $"f(x, y)"$ into a head $"f"$ and a tuple of argument names. Returns $(call, ())$ if the string does not have call syntax.

Handles nested parentheses correctly, e.g., $"f(g(x, y), z)"$ splits into $("f", ("g(x, y)", "z"))$.

source

lambdify_functions

Tarang.lambdify_functions — Function

lambdify_functions(call::AbstractString, result::AbstractString)

Convert a math-style definition $"f(x, y)"$/$"x*y"$ into a Julia anonymous function encoded as a string $"(x,y) -> x*y"$. Returns the original result for non-call statements to preserve standard semantics.

source

Reading NetCDF

Using NetCDF.jl

using NetCDF

# Read variable
data = NetCDF.ncread(filename, "temperature")

# Read attribute
attr = NetCDF.ncgetatt(filename, "NC_GLOBAL", "title")

# Read time array
times = NetCDF.ncread(filename, "t")

File Structure

NetCDF Layout

output_s1.nc
├── Dimensions
│   ├── x (128)
│   ├── z (64)
│   └── t (unlimited)
├── Coordinates
│   ├── x [128]
│   ├── z [64]
│   └── t [N_writes]
├── Variables
│   ├── temperature (t, x, z)
│   ├── velocity_x (t, x, z)
│   └── ...
└── Attributes
    ├── title
    ├── handler_name
    ├── software
    └── tarang_version

Global Attributes

Written automatically:

Attribute	Description
`title`	"Tarang.jl simulation output"
`handler_name`	Handler name from path
`software`	"Tarang"
`tarang_version`	Package version

Checkpointing

Saving State

function save_checkpoint(solver, filename)
    using JLD2

    state = Dict(
        "sim_time" => solver.sim_time,
        "iteration" => solver.iteration,
        "dt" => solver.dt,
        "fields" => Dict()
    )

    for (name, field) in solver.problem.fields
        Tarang.ensure_layout!(field, :c)
        state["fields"][name] = copy(get_coeff_data(field))
    end

    if MPI.Comm_rank(MPI.COMM_WORLD) == 0
        @save filename state
    end
end

Loading State

function load_checkpoint!(solver, filename)
    using JLD2
    @load filename state

    solver.sim_time = state["sim_time"]
    solver.iteration = state["iteration"]
    solver.dt = state["dt"]

    for (name, data) in state["fields"]
        field = solver.problem.fields[name]
        get_coeff_data(field) .= data
        field.current_layout = :c
    end
end

Parallel I/O Modes

Gather Mode

handler = add_netcdf_handler(path, dist, fields; parallel="gather")

All data gathered to rank 0
Single output file
Memory limited by rank 0

Virtual Mode

handler = add_netcdf_handler(path, dist, fields; parallel="virtual")

Each process writes own file
Filenames: output_p0.nc, output_p1.nc, etc.
Requires post-processing to merge

File Management

File Naming

Files are numbered sequentially:

output_s1.nc   # First file
output_s2.nc   # After max_writes reached
output_s3.nc   # etc.

Merging Files

For post-processing virtual files:

# Merge all parallel files
merge_netcdf_files("output"; output_file="output_merged.nc", cleanup=true)

# Batch merge multiple handlers
batch_merge_netcdf(["snapshots", "analysis", "checkpoints"])

Performance Tips

Batch writes: Don't write every iteration
Use gather for small outputs: Simpler file handling
Use virtual for large outputs: Better scaling
Compress if needed: NetCDF supports compression