Release Notes¶
v0.6.0 (Unreleased)¶
Changed
master_clock
,master_clock_dim
andmaster_clock_coords
tomain_clock
,main_clock_dim
andmain_clock_coords
and all occurences ofmaster
tomain
in the rest of the codebase. allmaster...
API hooks are still working, but raise a FuturewarningAdded access to main clock in initialize step as
main_clock_values
and as axr.DataArray
:main_clock_array
. for refering to the main clock as a dimension label, the placeholderxs.MAIN_CLOCK
can be used. This will be set to the main clock when storing the dataset.Changed default
fill_value
in the zarr stores to maximum dtype value for integer dtypes andnp.nan
for floating-point variables.
v0.5.0 (26 January 2021)¶
Breaking changes¶
Fill values are now masked (NA) when when loading the simulation output store as a xarray Dataset (#148). Note that Zarr sets the fill value to
0
by default for numeric data types, so it is highly recommended to explicitly define another fill value in model variable encodings if0
is expected to be a valid (non-missing) data value, or alternatively usemask_and_scale=False
in thedecoding
options passed toxarray.Dataset.xsimlab.run()
.
Enhancements¶
The
process()
decorator can now be applied on a class already decorated withattr.s()
(#150), usingapply_attrs=False
.Added
%create_setup
IPython (Jupyter) magic command to auto-generate code cells with a new simulation setup from a given model (#152). The command is available after executing%load_ext xsimlab.ipython
.Added an optional cache for on-demand variables (#156). The
@compute
decorator now has acache
option (deactivated by default).Added
group_dict()
variable (#159).Added
global_ref()
variable for model-wise implicit linking of variables in separate processes, based on global names (#160).Added
RuntimeSignal
for controlling simulation workflow from process runtime methods and/or runtime hook functions (#161).
Bug fixes¶
Fix saving output variables with dtype=object (#145).
Fix issues when saving output datasets to disk that were caused by the
_FillValue
attribute (#148).Ensure that classes given in a Model are all process-decorated, even those which inherit from process-decorated classes (#149).
Fix
update_clocks()
when only the master clock is updated implicitly (#151).Ensure that the
finalize
simulation stage is always run, even when an exception is raised during the previous stages (#154).
v0.4.1 (17 April 2020)¶
Enhancements¶
Added
xsimlab.Model.cache
public property (#125).Parameter
input_vars
ofcreate_setup()
andxarray.Dataset.xsimlab.update_vars()
now accepts array-like values with no explicit dimension label(s), in this case those labels are inferred from model variables’ metadata (#126).Single-model parallelism now supports Dask’s multi-processes or distributed schedulers, although this is still limited and rarely optimal (#127).
Improved auto-generated docstrings of variables declared in process classes (#130).
v0.4.0 (7 April 2020)¶
This is a big release which includes many exciting new features built on top of great Python libraries. It is now possible to set default, validate or convert model input values just like regular attributes in attrs, save model input/outputs with zarr, run model(s) in parallel using dask, monitor model runs with a tqdm progress bar, and much more!
Breaking changes¶
Python 3.6 is now the oldest supported version (#70).
The keys of the dictionary returned by
xarray.Dataset.xsimlab.output_vars
now correspond to variable names, and the values are clock dimension labels orNone
(previously the dictionary was formatted the other way around).xarray.Dataset.xsimlab.output_vars_by_clock
has been added for convenience (#85, #103).
Depreciations¶
Using the
group
parameter inxsimlab.variable()
andxsimlab.on_demand()
is depreciated; usegroups
instead.Providing a dictionary with clock dimensions or
None
as keys tooutput_vars
inxarray.Dataset.xsimlab.update_vars()
andxsimlab.create_setup()
is depreciated. Variable names should be used instead (#85).
Enhancements¶
It is now possible to assign multiple groups to a single variable (#71).
The xarray interface may now handle default values that are defined in model variables (#72). A new method
xarray.Dataset.xsimlab.reset_vars()
allows to (re)populate an input Dataset with variables and their default values.create_setup()
has also a newfill_default
parameter.Added static variables, i.e., variables that don’t accept time-varying input values (#73).
Added support for the validation of variable values (given as inputs and/or set through foreign variables), reusing
attr.validate()
(#74, #79). Validation is optional and is controlled by the parametervalidate
added toxarray.Dataset.xsimlab.run()
.Check or automatically transpose the dimensions of the variables given in input xarray Datasets to match those defined in model variables (#76). This is optional and controlled by the parameter
check_dims
added toxarray.Dataset.xsimlab.run()
.More consistent dictionary format for output variables in the xarray extension (#85).
%-formatting and str.format() code has been converted into formatted string literals (f-strings) (#90).
foreign()
has been updated so that it sets its description and its metadata from the variable it refers to (#91, #107).The
autodoc
parameter of thexsimlab.process()
decorator now allows to automatically add an attributes section to the docstring of the class to which the decorator is applied, using the metadata of each variable declared in the class (#67).Added
in_bounds()
andis_subdtype()
validators (#87).xsimlab.variable()
has now aconverter
parameter that can be used to convert any input value before (maybe) validating it and setting the variable (#92).Added
xsimlab.index()
for setting index variables (e.g., coordinate labels). Using the xarray extension, those variables are automatically added in the output Dataset as coordinates (#94).Added simulation runtime hooks (#95). Hooks can be created by using either the
runtime_hook()
decorator or theRuntimeHook
class.Added some useful properties and methods to the
xarray.Dataset.xsimlab
extension (#103).Added
ProgressBar
to track simulation progress (#104, #110).Added the ability to easily run batches of simulations using the
batch_dim
parameter ofxarray.Dataset.xsimlab.run()
(#115).Added ‘object’ variables
any_object()
for sharing arbitrary Python objects between processes (#118).Run one or multiple simulations in parallel using Dask (#119).
Bug fixes¶
v0.3.0 (30 September 2019)¶
Breaking changes¶
It is now possible to use class inheritance to customize a process without re-writing the class from scratch and without breaking the links between (foreign) variables when replacing the process in a model (#45). Although it should work just fine in most cases, there are potential caveats. This should be considered as an experimental, possibly breaking change.
Model.initialize
,Model.run_step
,Model.finalize_step
andModel.finalize
have been removed in favor ofModel.execute
(#59).
Depreciations¶
run_step
methods defined in process classes won’t accept anymore current step duration as a positional argument by default. Use theruntime
decorator if you need current step duration (and/or other runtime information) inside the method (#59).
Enhancements¶
Ensure that there is no
intent
conflict between the variables declared in a model. This check is explicit at Model creation and a more meaningful error message is shown when it fails (#57).Added
runtime
decorator to pass simulation runtime information to the (runtime) methods defined in process classes (#59).Better documentation with a minimal, yet illustrative example based on Game of Life (#61).
A class decorated with
process
can now be instantiated independently of any Model object. This is very useful for testing and debugging (#63).
v0.2.0 (9 May 2018)¶
Highlights¶
This release includes a major refactoring of both the internals and the API on how processes and variables are defined and depends on each other in a model. xarray-simlab now uses and extends attrs (#33).
Also, Python 3.4 support has been dropped. It may still work with that version but it is not actively tested anymore and it is not packaged with conda.
Breaking changes¶
As xarray-simlab is still at an early development stage and hasn’t been adopted “in production” yet (to our knowledge), we haven’t gone through any depreciation cycle, which by the way would have been almost impossible for such a major refactoring. The following breaking changes are effective now!
Variable
,ForeignVariable
andVariableGroup
classes have been replaced byvariable
,foreign
andgroup
factory functions (wrappers aroundattr.ib
), respectively.VariableList
has been removed and has not been replaced by anything equivalent.DiagnosticVariable
has been replaced byon_demand
and thediagnostic
decorator has been replaced by the variable’scompute
decorator.The
provided
(bool
) argument (variable constructors) has been replaced byintent
({'in', 'out', 'inout'}
).The
allowed_dims
argument has been renamed todims
and is now optional (a scalar value is expected by default).The
validators
argument has been renamed tovalidator
to be consistent withattr.ib
.The
optional
argument has been removed. Variables that don’t require an input value may be defined using a special validator function (seeattrs
documentation).Variable values are not anymore accessed using three different properties
state
,rate
andchange
(e.g.,self.foo.state
). Instead, all variables accept a unique value, which one can get/set by simply using the variable name (e.g.,self.foo
). Now multiple variables have to be declared for holding different values.Process classes are now defined using the
process
decorator instead of inheriting from aProcess
base class.It is not needed anymore to explicitly define whether or not a process is time dependent (it is now deducted from the methods implemented in the process class).
Using
class Meta
inside a process class to define some metadata is not used anymore.Model.input_vars
now returns a list of(process_name, variable_name)
tuples instead of a dict of dicts.Model.input_vars_dict
has been added for convenience (i.e., to get input variables grouped by process as a dictionary).Model.is_input
has been removed. UseModel.input_vars
instead to check if a variable is a model input.__repr__
has slightly changed for variables, processes and models. Process classes don’t have an.info()
method anymore, which has been replaced by theprocess_info()
top-level function. Another helper functionvariable_info()
has been added.In
Model.visualize()
andxsimlab.dot.dot_graph()
,show_variables=True
now shows all model variables including inputs. Items of group variables are not shown anymore as nodes.Model.visualize()
andxsimlab.dot.dot_graph()
now only accept tuples forshow_only_variable
.For simplicity,
Dataset.xsimlab.snapshot_vars
has been renamed tooutput_vars
. The corresponding arguments increate_setup
andDataset.xsimlab.update_vars
have been renamed accordingly.Values for all model inputs must be provided when creating or updating a setup using
create_setup
orDataset.xsimlab.update_vars
. this is a regression that will be fixed in the next releases.Argument values for generating clock data in
create_setup
andDataset.xsimlab.update_clocks
have changed and are now more consistent with how coordinates are set in xarray. Additionally,auto_adjust
has been removed (an error is raised instead when clock coordinate labels are not synchronized).Scalar values from a input
xarray.Dataset
are now converted into scalars (instead of a 0-d numpy array) when setting input model variables during a simulation.
Enhancements¶
The major refactoring in this release should reduce the overhead caused by the indirect access to variable values in process objects.
Another benefit of the refactoring is that a process-decorated class may now inherit from other classes (possibly also process-decorated), which allows more flexibility in model customization.
By creating read-only properties in specific cases (i.e., when
intent='in'
), theprocess
decorator applied on a class adds some safeguards to prevent setting variable values where it is not intended.Some more sanity checks have been added when creating process classes.
Simulation active and output data r/w access has been refactored internally so that it should be easy to later support alternative data storage backends (e.g., on-disk, distributed).
Added
Model.dependent_processes
property (so far this was not in public API).Added
Model.all_vars
andModel.all_vars_dict
properties that are similar toModel.input_vars
andModel.input_vars_dict
but return all variable names in the model.input_vars
andoutput_vars
arguments ofcreate_setup
andDataset.xsimlab.update_vars
now accepts different formats.It is now possible to update only some clocks with
Dataset.xsimlab.update_clocks
(previously all existing clock coordinates were dropped first).
Regressions (will be fixed in future releases)¶
Although it is possible to set validators, converters and/or default values for variables (this is directly supported by
attrs
), these are not handled by xarray-simlab yet.Variables don’t accept anymore a dimension that corresponds to their own name. This may be useful, e.g., for sensitivity analysis, but as the latter is not implemented yet this feature has been removed and will be added back in a next release.
High-level API for generating clock coordinate data (i.e.,
start
,end
,step
andauto_adjust
arguments) is not supported anymore. This could be added back in a future release in a cleaner form.
v0.1.1 (20 November 2017)¶
v0.1.0 (8 October 2017)¶
Initial release.