CHANGELOG.txt

Details of changes made to refnx
================================

0.1.1
-----
- removal of Python 2.7 support
- added azure pipelines for faster windows CI
- remove `uncertainties` as a package dependency (it's still used for testing)
- remove `six` as a package dependency
- add `refnx.util.refplot` for quick plotting of reflectometry datasets
- fixed various deprecation warnings
- added the ability to mask (hide) points in a `refnx.dataset.Data1D` dataset

0.1.2
-----
- added save/load model buttons in the interactive reflectometry modeller.
- removed `from __future__ import ...` statements as refnx is now solely Py3
- added `cython: language_level=3` statements to cython code
- marked cython extensions as c++ language
- smarter (faster) cataloging of NeXUS files
- removed pandas as a strict dependency, as it's only required for reduction,
  not analysis
- improved documentation of the ManualBeamFinder
- adding a pyqt based GUI (alpha state)
- improved __repr__ of many classes
- Start the GUI via a 'refnx' console command, via a setup.py entry_point.

0.1.3
-----
- GUI machinery can now use Components other than Slab, such as LipidLeaflet
  (already added). New Components may require extra shim code to be written for
  them. Specifically how they're to be displayed, and a default way of
  initialising the Component (which may require a dialogue).
- The 'Link equivalent parameters' action has been added, enabling equivalent
  parameters on other datasets to be linked. This greatly aids setup of multiple
  contrast datasets. All the datasets to be linked must have the same type of
  structure.
- The initialisation of a LipidLeaflet is made much easier by using a library
  of lipid head/volumes and scattering lengths for popular lipids which are
  presented to the user in an initialisation dialogue.
- The refnx paper is accepted and the article and manuscript file are
  included in the repository.
- The pyqt GUI to refnx can be made into standalone executables for Windows,
  macOS.
- Fixed a bug that meant most reflectivity calculations were single-threaded
  instead of multi-threaded.
- Added MixedReflectModel to the pyqt GUI, allowing one to model 'patchy'
  systems, i.e. incoherent averaging of reflectivities.
- BACKWARDS INCOMPATIBLE CHANGE: the slabs properties of `Component` and
  `Structure` have now been changed to methods, taking the optional `structure`
  kwd parameter. The reason for this is so each `Component` knows what kind of
  `Structure` it is in.
- The Spline Component can be used within the pyqt GUI.
- In the pyqt gui Components can be re-ordered within their host structure by
  drag/drop. Dragging to other Structures copies the Component to that
  structure.
- Added the Stack Component. A Stack contains a series of Components, and the
  Stack.repeats attribute can be used to produce a multilayer repeat structure.
- Folded in a reduction pyqt gui for Platypus data. The app was already in the
  slim directory. It's now available from refnx.reduce.gui

0.1.4
-----
- fixed bug in reflectivity calculation with significant non-zero absorption.
  The wavevector calculation was using the wrong branch of the complex sqrt.

0.1.5
-----
- fixed font size in pyqt GUI.
- script export from pyqt GUI can use either multiprocessing or MPI for
  parallelisation during MCMC sampling.
- speeded up reflectivity calculation, following on from changes made in 0.1.4
  (if the imaginary part of a complex sqrt argument is very small, then the C++
  calculation takes a lot longer).
- added a plot method to PlatypusNexus
- refactor util.PoolWrapper to util.MapWrapper
- allow the number of Stack repeats to be fittable.
- GUI option to only display those parameters (and datasets) that are going to
  be varied in a fit.
- update testimonials.bib
- "to code" button in Jupyter interactive modeller respects the transform
  popup.

0.1.6
-----
- When parameters are linked in the refnx GUI only the dataset containing the
  master parameter was being updated (reflectivity/SLD curves) when the master
  was changed. Now all datasets that have parameters linked to the master
  parameter (a constraint) are updated.
- When a dataset/component/structure containing a master parameter (i.e. a
  parameter to which other parameters are constrained to) is removed, the GUI
  now unlinks those dependent parameters from the master parameter.
- display number of iterations in GUI progress box. If fit is aborted put best
  fit so far into Objective that's being fitted.
- fixed crash resulting from the use of a comma when entering a floating point
  number. Entering '3,1' would crash the gui, using '3.1' would work but be
  displayed as '3,1'. The use of a dot as a decimal point is now enforced.

0.1.7
-----
- print human readable output when fitting with the Jupyter interactive
  modeller.
- added shgo and dual_annealing minimiser options to the refnx gui.
- SLD calculator retains state between viewings.
- Added dialogue to adjust optimisation parameters.
- Fixed bug in export of MCMC code fragment, Gui would crash.
- Autocorrelation plot produced from mcmc.py code fragment, this can be used to
  judge how much to thin the chains by.
- `refnx.analysis` now possesses a standalone function, `autocorrelation_chain`
  for calculating chain autocorrelation. Previously the calculation had to be
  done using a CurveFitter instance.
- Function for calculating autocorrelation time made visible as:
  `refnx.analysis.integrated_time`. You should pass the autocorrelation array
  to this function.
- GUI can now export an ASCII file representing the model SLD curve.
- BUG: when the GUI saves a model it should pickle a ReflectModel. It wasn't
  doing this, it was pickling a DataObject. This has now been fixed, but the
  fix affects back compatibility.
- *GUI can now do MCMC sampling*
- Add option to change context of Mapwrapper (spawn/fork/forkserver)
- BUG: when loading (not refreshing) a dataset that was already loaded, the
  associated model was lost. (GH331)
- Added a progress bar for batch fitting.
- Speeded up batch fitting

0.1.8
-----
- When GUI experiment file is loaded the correct fitting algorithm wasn't being
  set correctly (GH338).
- Prevent crash when trying to refresh a stale dataset (i.e. is no longer in
  its original location.
- Produce autocorrelation graph when sampling in GUI.
- Added links to ptemcee and emcee in the optimisation parameters window.
- GUI produces corner plot after MCMC sampling
- BUG fix for rebinning code (doesn't affect analysis)
- Different interfacial roughness types can be specified between all Components
  in a Structure. The available types are: Erf (Error Function, default), Tanh,
  Sinusoidal, Exponential, Step, Linear. User specifiable interfaces can be
  created by subclassing Interface.
- A fix for the PyQt5 interface using 4K screens on Windows was made.
  Previously the GUI elements and fonts were being displayed in a much too
  small fashion.
- Made various dialogues in the PyQt5 GUI window modal (to prevent them being
 lost).
- fixed bugs if cancel was pressed during the MCMC folder dialogue phase.
- made loading of experiment file back compatible.

0.1.9
-----
- Added DOTAP, h-DOPC, 18:1 Diether PC to lipids database
- Updated SLIM reduction software to cope with the new monobloc detector
- Improve speed of resolution smearing by using splev/splrep
- Added shell scripts to build manylinux and macOS wheels (as well as test
 them).
- Some minor optimisations for calculation of `Interval.logp`.
- Some minor optimisations for various calculations in `Objective`.
- Fixed potential for crashes in ManualBeamFinder if controls specified
  regions that went outside the detector region.
- cythonized contract_by_area, resulting in huge speedup for microsliced
  structures.
- Enabled parallel calculation of reflectivity using OpenMP. Tests show that
  it should be ~20% faster than the previous calculation in C.
- tqdm progress bar for ptemcee sampling (if tqdm is installed).
- Cleaned up the _creflect module. Threaded reflectivity calculation in that
  extension now uses std::thread instead of pthreads (POSIX) or WinAPI
  (windows).

0.1.10
------
- Prevent spurious benchmark package installation.
- Fixed bug when loading a MTFT file saved in a previous version of refnx.
- event mode data reduction sped up by an order of magnitude
- Added neutron transmission calculator (if periodictable is present)
- event file reader can now read any ANSTO packedbin file (reduction).
- Align SLD plots around a specific interface in a slab representation. Useful
  if plotting many samples at the same time.
- Add MaterialSLD object that is constructed from a chemical formula and mass
  density. This enables the use of specific materials to describe layers, e.g.
  MaterialSLD('SiO2', density=2.2).
- Components can be multiplied by an integer to make them repeat.
- Add _open_mp_helpers to MANIFEST.in (gh381).
- update bundled vendored emcee

0.1.11
------
- Fixed a bug in the GUI that prevented load/save of mixed area models.
- Fixed a bug in the resynthesis of data.
- Fixed a bug in the loading of event data from the Platypus monobloc detector.
- Added a FresnelTransform.
- Document inequality constraints.
- Made refnx SLD calculator GUI more fault tolerant to incorrect formula.
- Add example Jupyter notebook for batch reduction.
- Autoscaling for Panalytical XRR reduction.
- Fixed bug in drag/drop within a Stack in the GUI.
- Added a MixedSlab Component which is constituted from several individual
  Scatterers. The SLD of the MixedSlab is weighted by their volume fractions.

0.1.12
------
- allow master and slave chopper parameters to be ignored in PLP reduction.
- tunable t_0 offset in PLP/SPZ reduction.
- possibly_create_parameter accepts default bounds/vary/constraint. This fixes
  a bug in MixedSlab.
- Don't require numpy be installed before setup.py can run.
- Speed up reflectivity calculation if there is solvation.
- reflect.choose_dq_type for finding out fastest mode of resolution smearing.
- User can now select resolution smearing approach in ReflectModel. Choosing
  between 'pointwise' or 'constant'.
- Add progress bar for Curvefitter.fit() (requires tqdm being installed).
- A few micro-optimisations.
- Optimized pickling/unpickling of Bounds instances. This can lead to a huge
  performance increase (~ 40%) when doing parallel sampling.
- MCMC sampling initialisation made reproducible

0.1.13
------
- synthesising of datasets (Data1D.synthesise) can now be repeatable by
  providing a seed.
- use Github Actions to test and build macOS wheels across all the Pythons.
- The _creflect.abeles reflectivity calculation can release the GIL for a
  large part of its calculation now. This will enable parallelisation using
  either Processes or Threads. Processes still have the edge on speed at the
  moment.
- change reduction code for ReflectNexus.phase_angle to use degrees, not
  radians. (previously angles were returned in a mix of degrees/radians)
- Speed of reflectivity calculation is improved between 10 and 20 %. This is
  achieved by the use of C99 complex arithmetic instead of C++ std::complex.
  This improvement does not apply to Windows because it doesn't have a
  C99 standard conformant compiler.
- Document how to save a model.
- macOS wheels (and CI testing) have the cyreflect openmp option activated.
- An openCL reflectivity calculation backend is added that can use a GPU.
- Added a `reflect.use_reflect_backend` function to choose between the backends
  used to calculate reflectivity.

0.1.14
------
This will be a bug fix release over 0.1.13. The Linux wheels weren't tested
enough and the default reflectivity calculation backend didn't work correctly
in 0.1.13.
- fix default backend calculation.

0.1.15
------
- Build standalone GUI as part of continuous integration runs.
- Fix couple of GUI warnings when started from a terminal.
- pyinstaller plist settings adjusted to make retina compatible and include
  refnx version number.
- use Github Actions to test and build Linux wheels across all the Pythons.
- Compensate for dq_type being added to ReflectModel (older .mtft files
  wouldn't load into GUI).
- Refactor test_reflect to challenge all reflectivity calculation backends.
- refnx.reflect.available_backends lists all reflectivity calculation backends.
- Add pytest fixture to download test data. This dramatically reduces the
  size of the package.
- Compensate for _stderr being added to Parameter (older .mtft files wouldn't
  load properly into GUI).
- Fixed progress bar for parallel tempered sampling.
- Make sdist

0.1.16
------
- build more Linux wheels.
- vendored ptemcee (bleeding edge), so it's no longer necessary to install it.
- use -funsafe-math-optimizations for reflectivity calculation.
- fixed bug when retrieving log-probabilities from PTMCMC sampling.
- fixed encoding call for Py3 (.decode doesn't exist for Py3).
- improve pymc3 model creation. pymc3 offers different ways to carry out MCMC
  sampling. This may lead to pymc3 being included as a sampling method in
  Curvefitter.
- implement `invcdf` method for Bounds objects. This will allow creation of
  prior transforms from [0, 1) to the original range specification.
- BREAKING CHANGE: `objective.model.logp` and `Objective.logp_extra` are now
  included in the log-likelihood (`Objective.logl`) instead of log-prior
  (`Objective.logp`). This makes it easier to work with pymc3 and dynesty.
- Add `Objective.prior_transform` that converts random variates in a unit
  hypercube to parameter values, according to their prior distributions.
- manual_beam_find can be sent a name to be displayed on the window title.
- ReductionOptions dict is now used for specifying options for reduction and
  processing of reflectometry data. This will permit different settings to be
  used for datasets measured in different configurations.
- Add AutoReducer, an object that watches a directory for modified/created
  files and automatically reduces them.

0.1.17
------
- Drop Python 3.6 for new development (as per
  https://numpy.org/neps/nep-0029-deprecation_policy.html)
- Pin setuptools version on RTD to allow the docs to build.
- Add a tutorial to the docs on model selection.
- Update emcee vendored code.
- Fix repr of Model
- Fix several warnings during test run.
- Add error bar plotting to refnx GUI.
- Improved the specular ridge finding methodology for data reduction.

0.1.18
------
- SPZ batch reduction
- remove some roadblocks to JAXification (float casting in Parameter.value
 setter)
- volume fraction of solvent can now be used when constructing a Slab from a
 Scatterer.
- Fully remove traces from graph when opening new experiments.
- Store reduction_options when processing spectra.
- RuntimeError if a log transform creates non-finite numbers.
- ReflectModels can now use a q_offset to correct for possible angular
  misalignments.
- `possibly_create_parameter` and `is_parameter` now use `BaseParameter` for
  their operation, rather than `Parameter`. Thus _BinaryOp and _UnaryOp can be
  given to these operations and work. This enables construction of various
  objects (SLD/Slab/etc) from constrained parameters, rather than setting a
  constraint afterwards.
- MixedSlab is now allowed to be the first/last Component in a Structure.

0.1.19
------
- MCMC sampling in GUI now plots parameter value vs step number, so you can
  see how many samples to burn
- Fixed GUI crash when trying to make a snapshot with a name that has already
  been used.
- Give BaseObjective a default weighted attribute, enabling it to be used in
  GlobalObjective.
- Fallback to numerical estimation of Hessian/Covariance matrix if an
  Objective cannot calculate a residuals array (e.g. mixed use of
  BaseObjective/Objective in a GlobalObjective).
- Vendor scipy.stats.qmc.
- Use Latin Hyper Cube sampling to initialise MCMC walkers when initialising
  with prior. This ensures a good distribution of walkers and no clustering.

0.1.20
------
- Make default Spline.dz.bounds.lb > 0, so that crashes aren't experienced
  when a dz is set to 0. GH549
- Add the shgo keyword parameters 'n' and 'iters'.
- Add max_delta_z to sld_profile so that the point density in an SLD plot
  can be changed.
- Don't use tqdm progress bar when fitting in GUI if sys.stderr is None.

0.1.21
------
- check that refnx/analysis doesn't get in the way of handling
  multidimensional data.
- add a header to datafiles during reduction.
- make requirements.txt available in the refnx GUI.
- add units attribute to Parameter. This is displayed as a tooltip in the GUI.
- GUI and refnx command line entry point now reads sys.argv to see if GUI
  should open an experiment file.
- Associate the mtft file extension on macOS with the GUI app, meaning you can
  double click on a mtft file and it'll open with the app. The mtft files also
  display with the GUI icon.
- BUG: an error was experienced whilst reprocessing an existing chain in the
  refnx GUI.
- BUG: fix gh336, remember the dq/q choice on reloading an experiment file.
  Previously the GUI reverted to constant dq/q on a reload, even if it was
  deselected.

0.1.22
------
- Added polarisation reduction (Oliver Paull)
- Introduce Objective.auxiliary_params. These are extra parameters that are
  modified during the calculation of Objective statistics, and are varied
  during a fit, but may not be directly part of Model calculating a signal.
  Their main purpose is to aid in creating constraints.
- Add example to show how to co-refine non-spinflip NR data.
- Silenced warning during test from slow reflectivity calculation, which
  erroneously gave the impression that the C based kernel wasn't available.

0.1.23
------
- bugfix: remove printing of auxiliary parameters in Objective.setp. A remnant
  print function was in that method. However, those parameters aren't present
  in a global objective, which caused an AttributeError.

0.1.24
------
- use oldest-supported-numpy to ensure that older numpy versions are used to
  build refnx wheels. This will allow refnx to be used with more recent
  versions as well as the oldest version supported on a given Python version.
  Previously refnx wheels were built with the newest numpy available at
  build time, and were probably not compatible with older numpy versions.
- Rejig the build process to make macOS universal wheels.

0.1.25
------
- Add NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION define to cython extension
  building to remove warnings. This requires a different mode of access to
  the array data.
- Fixed drag and drop of datasets onto refnx GUI.
- Add OrsoDataset and a generalised load_data function.
- refnx GUI removes horizontal scrollbar.
- Fixed crash when trying to access version numbers when saving expt file.

0.1.26
------
- change wheel builder to make 3.10 wheels.
- check that pyinstaller based macOS standalone app runs in CI.
- update install documentation
- workaround for difficulty using orsopy on Python 3.10
- added fields to ReflectReduce output dict, including omega, reflected_beam,
  direct_beam.

0.1.27
------
- Add a little machinery for energy dispersive calculations. This is
  implemented by adding a wavelength attribute to Structure. The
  Structure.slabs method then works out the Slab representation for that
  particular wavelength. The wavelength attribute should be used by the
  Component.slabs method when asking for the SLD of a Scatterer. The Slab and
  MixedSlab Components have been altered to permit this. User provided
  components should work as they currently do, but probably have to be
  rewritten to permit energy dispersive operation.
  For energy dispersive operation classes that inherit Scatterer should
  override the newly introduced Scatterer.complex() method.

0.1.28
------
- add name="refnx" to pyproject.toml.

0.1.29
------
- Don't pickle the manual_beam_finder.
- Fixed a bug when inserting a Component into a Stack in the refnx GUI. If
  it was appended to the end of the Stack Component addition worked.
  Inserting a Component into the middle of the Stack appeared to put it in the
  correct place in the GUI, but inserted it in the wrong place in the
  Structure.
- Added Lagrangian multipliers to GlobalObjective, allowing the user to
  modulate the contribution of individual log-likelihoods to the whole. This
  may be of use when co-refining Neutron/X-ray/Ellipsometry datasets.

0.1.30
------
- First appearance of Parameter.set_constraint().
- Previously Parameter constraints were limited to algebraic Python
  expressions. Now Parameter.set_constraint can be given a callable which
  is evaluated at runtime, allowing more sophisticated constraints to be
  developed. (N.B this is separate to the inequality constraints that can
  be made during curvefitting using the NonLinearConstraint machinery).
- Add a Parratt recursion for calculating reflectivity, it seems to be a
  little faster.
- Ported the pymc wrapper from pymc3 --> pymc (v4), which was released in
  Jun2022.
- added sequence_to_parameters function.
- added LipidLeaflet.make_constraint.

0.1.31
------
- Amend setup.py package discovery
- Change the manual beam finder box to green
- When a Parameters object contains a _UnaryOp or _BinaryOp the dependencies
  of those objects are queried for varying parameters when asking for
  Parameters.varying_parameters()

0.1.32
-------
- event mode reduction for SPZ
- Display SLD profile for first structure in MixedReflectModel
- Github: changed the default branch from master to main. This is done to
  match other repositories behaviour and to reduce cognitive load.
- Added a Parameter.corner method to plot the posterior distribution for a
  given Parameter.