ReleaseNotes.html

<h1>Version 0.6.6 - 2016-07-15</h1>
<h2>Incompatibilities</h2>
<h3>Changes in <code>IPython</code>-notebooks 3.0</h3>
<p>With IPython 3.0 the names of the Widget classes lost the <code>Widget</code> in the name (for instance <code>DropdownWidget</code> becomes <code>Dropdown</code>). <code>PyFoam</code> has been changed to conform with this but as a consequence won't work with old version of the <code>IPython</code> notebooks</p>
<h2>Enhancements to Utilities</h2>
<h3>=pyFoamPrepareCase.py= executes <code>setFields</code> if appropriate</h3>
<p>If no setup-script is specified and if there is a <code>setFieldsDict</code> present then <code>setFields</code> is automatically executed</p>
<h3>Plotting utilities now automatically add custom plots depending on the solver name</h3>
<p>The configuration system now has a section <code>Autoplots</code>. The entries there are all dictionaries with two entries:</p>
<dl>
<dt>solvers</dt>
<dd>a list of regular expressions. If the name of the currently used solver matches any of these expressions then this entry is used
</dd>
<dt>plotinfo</dt>
<dd>a dictionary with the plot information known from regular <code>customRegexp</code>: <code>expr</code>, <code>titles</code> etc
</dd>
</dl>
<p>If the solver matches <code>solvers</code> then a custom plot based on <code>plotinfo</code> is automatically added.</p>
<p>If the solver does not fit the plot is <strong>not</strong> added (this helps avoid processing of unused <code>expr</code>. This is important because processing the <code>expr</code> is one of the things <code>PyFoam</code> uses the most computation time for)</p>
<p>Some common plots (phase fractions for instance or most of the output of <code>chtMultiRegionFoam</code>) are hardcoded into <code>PyFoam</code></p>
<p>Also there is a new section <code>[SolverBase]</code> in the configuration. It is checked whether the current solver name <strong>begins</strong> with any of the keys there. If it does then the list of solvers is assumed to be the solvers this solver was based on and <code>Autoplots</code> for those solvers are added as well. For instance</p>
<pre class="example"><code>[SolverBase]
myReactingCht: [&quot;chtMultiRegionFoam&quot;,&quot;reactingFoam&quot;]
</code></pre>
<p>assumes that a solver <code>myReactingChtFoam</code> is based on these two solvers and automatically adds their <code>Autoplots</code></p>
<p>In addition if a setting like</p>
<pre class="example"><code>[Plotting]
autoplots: cloudNumberMass
</code></pre>
<p>is set (for instance through <code>LocalConfigPyFoam</code> in the case) then the autoplot <code>cloudnumbermass</code> is used regardless of the solver name</p>
<h3>=alternateAxis=-entries now can be regular expressions</h3>
<p>This allows specifying plots generated with <code>type dynamic</code> on the alternate axis</p>
<h3>Plotting utilities now allow choice of Gnuplot terminal</h3>
<p>These utilities now allow with the option <code>--gnuplot-terminal</code> to choose the terminal. Otherwise the terminal specified in the configuration (usually <code>x11</code>) is used</p>
<h3>Plotting utilities now sort legend by name</h3>
<p>Names in the legend are now sorted. This improves readability for large numbers of lines in the plot</p>
<h3>=pyFoamExecute.py= allows calling with debugger</h3>
<p>The option <code>--run-with-debugger</code> runs the command in the debugger. The arguments are appropriately handled</p>
<h3>=pyFoamPrepareCase.py= fails if execution of a script fails</h3>
<p>If the execution of a script fails (the status code returned by the script is not <span class="math">0</span>) then execution of the utility fails (before the utility just continued). This behavior can be overridden with the option <code>--continue-on-script-failure</code>.</p>
<p>It is up to the script to ensure that the failure of a utility called in the script is forwarded to <code>pyFoamPrepareCase.py</code>. For instance with</p>
<pre class="example"><code>set -e
</code></pre>
<p>in a <code>bash</code>-script</p>
<h3>=--hardcopy= in plotting library now allows modification of <code>gnuplot</code>-terminals</h3>
<p>This option allows setting the options for the terminal selected with <code>--format-of-hardcopy</code>. This overrides any default set in configuration section <code>[Plotting]</code> under the name <code>hardcopyOptions_&lt;term&gt;</code> with <code>&lt;term&gt;</code> being the name of the terminal (for instance for <code>png</code> the option is <code>hardcopyOptions_png</code>.</p>
<h3>=pyFoamPrepareCase.py= writes state information about what it is currently doing</h3>
<p>It writes to the usual state file. This way <code>pyFoamListCases.py</code> will list this information. If the scripts call <code>pyFoamRunner.py</code> then this information will be overwritten</p>
<h3>=pyFoamBinarySize.py= can handle new location of binaries in OpenFOAM 3.0</h3>
<p>Since that foam version all binaries (and object files are located in the directory <code>platforms</code>. The utility now finds them there</p>
<h3>=Runner=-utilites now can signal on <code>blink(1)</code>-devices</h3>
<p>With the option <code>--use-blink1</code> these utilities now flash on a plugged in <code>blink(1)</code> USB-device for every time-step</p>
<h3>=pyFoamExecute.py= can flash a <code>blink(1)</code></h3>
<p>To indicate that the utility is still running it is able to play a pattern on a <code>blink(1)</code>-device. This is switched on with <code>-use-blink</code></p>
<h3>=pyFoamDecompose.py= allows using a template file</h3>
<p>With the option <code>--template-dict</code> it is possible to initialize the output with an existing file. With this file it is possible to add 'complicated' settings.</p>
<h3>=pyFoamTimelinePlot.py= now handles new format of probe files</h3>
<p>Probe files now have one comment line per probe to specify the position. This format is now correctly detected and plotted. Old probe files are also handled</p>
<h3>=ReST=-report of <code>pyFoamPrepareCase.py</code> now reports derived parameters</h3>
<p>The <code>.rst</code>-file written by the utility now adds a section on derived parameters if such parameters were specified in a script</p>
<h3>=pyFoamPrepareCase= can now ignore directories</h3>
<p>It is now possible to specify directories that should be ignored when looking for templates. Some sensible defaults like <code>postProcessing</code>, <code>processor*</code> and <code>VTK</code> are already set</p>
<h3>=pyFoamConvertToCSV.py= allows adding formulas to XLSX-files</h3>
<p>The option <code>--add-formula-to-sheet</code> allows adding formulas to the Excel-sheet. Something like</p>
<pre class="example"><code>--add-formula=&quot;massflow:::&#39;inlet&#39;-&#39;outlet&#39;&quot;
</code></pre>
<p>adds a column <code>massflow</code> that subtracts the columns <code>inlet</code> and <code>outlet</code></p>
<h3>=pyFoamListCases.py= now displays mercurial info</h3>
<p>For those who use mercurial to keep track of their cases the utility now has the option <code>-hg-info</code> that displays the mercurial hash-ID, the local id and the branch name</p>
<h3>Progress bar added to utilities with long run-time</h3>
<p>Using the library <code>lqdm</code> progress bars have been added to utilities that have a long run-time and where the progress bars are not disturbing the regular output. These utilities are</p>
<ul>
<li><code>pyFoamListCases.py</code></li>
<li><code>pyFoamBinarySize.py</code></li>
</ul>
<p>Bars can be switched off with <code>--no-progress-bar</code></p>
<h3>Utilities that clear data can now report what is cleared</h3>
<p><code>pyFoamCleasCase.py</code> and all utilities that have a <code>--clear</code> option now also have a <code>--verbose-clear</code> option that reports <strong>what</strong> is being cleared</p>
<h3>=pyFoamConvertToCSV.py= now allows manipulating the input</h3>
<p>The utility now has two new options:</p>
<dl>
<dt>-</dt>
<dd>This allows removing characters before the file is actually read
</dd>
<dt>-</dt>
<dd>Replaces the first line (for instance if the header does not match the data
</dd>
</dl>
<h2>Enhancements to the Library</h2>
<h3>Detection of <code>OpenFOAM-dev</code></h3>
<p>A development installation is now also detected and it is assumed that this uses the new calling convention. Also: PyFoam reports this as version <code>9.9.9</code> (as this is larger than any version in the foreseeable future</p>
<h3>Add <code>OpenFOAM+</code> as a fork</h3>
<p>An additional fork type <code>openfoamplus</code> has been added (in addition to <code>openfoam</code> and <code>extend</code>). Installations of the form <code>OpenFOAM-vX.X+</code> (with <code>X.X</code> being the version number) are added to this fork. Also <code>OpenFOAM-plus</code> is added as the development version of this fork</p>
<h3>Accept new convention for location of <code>blockMeshDict</code></h3>
<p>In newer OpenFOAM-versions <code>blockMeshDict</code> may be located in <code>system</code>. PyFoam detects it either there or in the old <code>constant/polyMesh</code>-location</p>
<h3>Handling of complex data by <code>Configuration</code></h3>
<p>Lists and dictionaries now can also be specified. Have to be correctly formatted if they are longer than one line (indented by at least one space - convention for configuration files)</p>
<h3>=Configuration= has method <code>getArch</code> for architecture dependent settings</h3>
<p>If an option <code>opt</code> is requested with this option then it is checked whether an architecture-dependent setting exists. Architecture <code>arch</code> is the output of the <code>uname</code>-command. The architecture-dependent name is <code>opt_arch</code>.</p>
<h3>=execute=-method from <code>PyFoam.Basics.Utilities</code> returns status-code</h3>
<p>This function now has an option that makes it return the status of the execution as well as the output of the execution.</p>
<h3>=BasicRunner= now supports more ways of stopping runs</h3>
<p>In the past this class (and the utilities based on it) looked for a file <code>stop</code> and stopped the run (with writing) if it was found. Now two additional files are looked for</p>
<dl>
<dt>stopWrite</dt>
<dd>this waits for the next scheduled write and then stops the run
</dd>
<dt>kill</dt>
<dd>gracefully stops the run without any writing
</dd>
</dl>
<h3>Added <code>Blink1</code> class to support <code>blink(1)</code> devices</h3>
<p>This class assumes that a <code>blink(1)</code> USB-device is present and the API-server (from the <code>Blink1Control</code>-program for this is running. It wraps these calls so that utilities can use them conveniently</p>
<h3>=ParsedParameterFiles= now supports <code>includeEtc</code></h3>
<p><code>#includeEtc</code> is now supported</p>
<h3>Parses uniform fields correctly</h3>
<p>Uniform fields of the form <code>1002{42.5}</code> (Field with 1002 values <span class="math">42.5</span>) are now correctly parsed</p>
<h3>=toNumpy=-method added to <code>Unparsed</code> and <code>Field</code></h3>
<p>These two classes have a method <code>toNumpy</code> added that transformed the data into a structured <code>NumPy</code>-array. There are no applications for this in <code>PyFoam</code> yet but an application will be the parsing of lagrangian data</p>
<h3>Added module <code>PyFoam.RunDictionary.LagrangianPatchData</code> to read data from patch function object</h3>
<p>This module reads data written by the cloud function-object that writes particle data as particles hit the patches and transforms it into <code>numpy</code>-array. Which can also be returned as <code>pandas</code> <code>DataFrames</code></p>
<p>It adds some properties to the data</p>
<ul>
<li>the patch name</li>
<li>the time at which this data was written</li>
<li>a <code>globalId</code> constructed from <code>origId</code> and <code>origProcId</code></li>
</ul>
<h3>Added module <code>PyFoam.RunDictionary.LagrangianCloudData</code> to read cloud data</h3>
<p>This gets</p>
<ul>
<li>a case</li>
<li>a cloud name</li>
<li>a time name and reads the lagrangian data from the specified time and converts it to a pandas <code>DataFrame</code></li>
</ul>
<p>A <code>globalId</code> that is consistent with the one in <code>LagrangianPatchData</code> is set</p>
<h3>Method <code>code</code> added to =RestructuredTextHelper</h3>
<p>This method formats a string assuming that it is a program code. Default value is <code>python</code></p>
<h3>=ParsedParameterFile= now parses new dimension format correctly</h3>
<p>Newer OpenFOAM-versions allow dimensions in symbolic format (for example <code>[ m s^-1 ]</code>). These are now correctly parsed</p>
<h3>=ParsedParameterFiel= now parses uniform fields correctly</h3>
<p>Fields of the form <code>23 { 4.2 }</code> (meaning &quot;23 times 4.2&quot;) are now correctly parsed</p>
<h2>Infrastructure</h2>
<h3>Change of documentation from <code>epydoc</code> to <code>sphinx</code></h3>
<p>As <code>expydoc</code> is discontinued the API-documentation is now generated with <code>sphinx</code>. Just run</p>
<pre class="example"><code>make docu
</code></pre>
<p>to do so.</p>
<p>Advantage is that now with</p>
<pre class="example"><code>make docset
</code></pre>
<p>a document set for offline searching with <code>Dash</code> (for Mac OS X) or clones (on other OSes) can be generated</p>
<h3>Adaptions to the unittests</h3>
<p>Untitests only used to run correctly if the OpenFOAM-version was 1.7. Are changed to run with OF 3.0. No effort has been made to support intermediate versions as the changes are mainly about changed tutorials</p>
<h2>Bug fixes</h2>
<h3>Wrong format of <code>ExecutionTime</code> breaks plotting utilities</h3>
<p>If the <code>ExecutionTime</code> is not as expected <code>pyFoamPlotWatcher.py</code> and <code>pyFoamPlotRunner.py</code> finish with an error. This is now more robust</p>
<h3>=phases= not working with dynamic plots</h3>
<p>For dynamic plots the addition of the phase name did not work. Fixed</p>
<h3>Phase name added to function object output</h3>
<p>If <code>phase</code> was set the output of the function objects got it added to the names even though the function objects do not belong to the phase. This is fixed</p>
<h3>One region mesh too many in utilities that change the boundary</h3>
<p>When working with regions one region too many was added in <code>pyFoamChangeBoundaryType.py</code> and <code>pyFoamChangeBoundaryName.py</code>. Fixed</p>
<h3>=pyFoamClearCase.py= fails on write-protected case</h3>
<p>If a case is write protected then the utility failed. Now it only issues a warning and continues cleaning</p>
<h3>Copying of directories in <code>pyFoamPrepareCase.py</code> confused by zipped files</h3>
<p>When copying one file to another and one of them is zipped then copying doesn't replace the destination correctly but adds the zipped/unzipped variant</p>
<h3>Wrong times for multi-view layouts in <code>pyFoamPVSnapshots.py</code></h3>
<p>If snapshots were taken from state files with multiple layouts then some of the views had the wrong time (either that from the state-file or from the timestep before). Fixed</p>
<h3>First timestep not plotted (and not stored)</h3>
<p>The data from the first timestep was not plotted under certain circumstances. This has been fixed</p>
<h3>=DYLD<sub>LIBRARYPATH</sub>= not passed on <em>Mac OS X 10.11</em></h3>
<p>Starting with this OS-version as a security feature the system does not pass <code>LD_LIBRARY_PATH</code> and <code>DYLD_LIBRARY_PATH</code> to a shell. <code>PyFoam</code> detects this and creates these variables and makes sure they are passed to the processes</p>
<h3>Newer versions of <code>pandas</code> broke the writing of excel files with <code>pyFoamConvertToCSV.py</code></h3>
<p>The reason is that the old way of making axis data unique did not work anymore. This has been fixed</p>
<h3>Capital <code>E</code> in exponential notation for floats breaks parser</h3>
<p>This problem has been reported at <a href="https://sourceforge.net/p/openfoam-extend/ticketspyfoam/220/">https://sourceforge.net/p/openfoam-extend/ticketspyfoam/220/</a> (the number <code>1E-2</code> is not correctly parsed to <code>0.01</code>) and has been fixed</p>
<h3>=Runner=-utilities clear processor directories if first time in parallel data differs</h3>
<p>In cases where the parallel data has a different start directory than <span class="math">0</span> the <code>pyFoamRunner.py</code> and similar utilities cleared that data and made a restart impossible. This has been fixed</p>
<h3>Utilities <code>pvpython</code> not working when installed through <code>distutils</code></h3>
<p>As the <code>distutils</code> (and all mechanisms built on these like <code>pip</code>) replace the used python in scripts the necessary <code>pvpython</code> was removed. This has been fixed by generating a temporary script file that is actually executed with =pvpython)</p>
<h2>ThirdParty</h2>
<h3>Added <code>tqdm</code> for progress bars</h3>
<p>Add the library <code>tqdm</code> (<a href="https://github.com/tqdm/tqdm">https://github.com/tqdm/tqdm</a>) for adding progress bars to utilities.</p>
<p>Library is under <code>MIT</code> License</p>
<h1>Version 0.6.5 - 2015-06-01</h1>
<h2>Major changes</h2>
<h3>PyFoam now on <em>Python Package Index</em></h3>
<p>PyFoam is now found at <a href="https://pypi.python.org/pypi/PyFoam">https://pypi.python.org/pypi/PyFoam</a></p>
<p>Recommended way of installing is using <a href="https://pip.pypa.io/en/latest/">https://pip.pypa.io/en/latest/</a> :</p>
<pre class="example"><code>pip install PyFoam
</code></pre>
<p>This will also make sure that the required <code>numpy</code>-package is installed</p>
<h2>Incompatibilities</h2>
<h3>=ArchiveDir= in <code>SolutionDirectory</code> discouraged</h3>
<p>As this was never really used it is discouraged (the option is still there).</p>
<p>If you don't understand what this means it probably doesn't concern you</p>
<h3>Pickled data files now written as binary</h3>
<p>All pickled files are now written and read in binary mode (as this was the only way that works consistently in Python 3). This <strong>may</strong> cause problems with old cases (but no effort has been made to check whether this problem actually exists)</p>
<h3>The <code>PlotRunner</code> and <code>PlotWatcher</code> now don't strip spaces</h3>
<p>These two utilities now don't strip leading spaces from the read lines. This preserves formatting in the output but might break scripts that rely on these spaces.</p>
<p>The old behaviour may be reset by overriding <code>stripSpaces</code> in section <code>SolverOutput</code> with a value <code>True</code></p>
<h3>Different column names in <code>pyFoamConvertToCSV.py</code></h3>
<p>The enhanced naming of the columns might break scripts that rely on the old naming</p>
<h3>=pyFoamChangeBoundaryName.py= and <code>pyFoamChangeBoundaryType.py</code> automatically modify <code>processorX</code></h3>
<p>In previous versions these boundary files were not modified. Now they are. Scripts that rely on unchanged <code>boundary</code>-files in the <code>processorX</code>-directories might fail. Old behavior can be set with the <code>--no-processor</code>-option</p>
<h2>Bugfixes</h2>
<h3>Arbitrary commands in <code>TemplateFile</code> passed to file</h3>
<p>Lines with <code>$$</code> are passed to the file and make it syntactically incorrect. Fixed</p>
<h3>Pickled files not opened in binary mode</h3>
<p>This caused problems in Python 3 were binary strings were not correctly written (actually: attempts to write them to a pickle-file made the application fail)</p>
<h3>Additional fixes for Python 3</h3>
<p>In a number of classes/applications semantic differences between Python 2 and 3 makes these fail on Python 3. Changes are</p>
<ul>
<li>Replace <code>map</code> with list comprehension where possible</li>
<li>Modify wrappings in <code>CTestRun</code> to work with Python3</li>
<li>Replace <code>print</code> with <code>print_</code></li>
<li>Semantic difference in division of two integers: Python3 gives a floating point number as a result</li>
</ul>
<h3>=ParsedParameterFile= fails if &quot;complete&quot; dictionary is <code>#include</code> ed</h3>
<p>If an included dictionary has a header parsing failed. This is fixed by retrying the parsing with the header</p>
<h3>=ParsedParameterFile= fails if there is more info after <code>#include</code></h3>
<p>If there is for instance a <code>;</code> after an <code>#include</code> statement the regular OpenFOAM-parser ignores it. The PyFoam-parser failed. This has been fixed and the parser behaves like regular OpenFOAM</p>
<h3>=pyFoamDisplayBlockMesh.py= not working with VTK 6</h3>
<p>Due to changes in the API the program did not work. This is now fixed and the program works with VTK 6 as well as VTK 5</p>
<h3>=pyFoamCreateModuleFile.py= failed with environment variables containing </h3>
<p>In that case an overeager <code>split</code> created lists.</p>
<p>Fix provided by Martin Beaudoin</p>
<h3>Fix import in <code>GeneralVCSInterface</code></h3>
<p>Change in the syntax of imports in Python 3 broke this one. Fixed. But doesn't matter as Mercurial doesn't support Python3</p>
<h3>Support of old format in <code>ParsedBlockMeshDict</code> broken</h3>
<p>Wrong usage of indexes. Fixed</p>
<h3>=TemplateFile= not correctly working in Python 3</h3>
<p>Reason was a different calling convention to the <code>exec</code>-function of Python. Fixed</p>
<h3>Certain things not done by <code>pyFoamPrepareCase</code> in <code>--quiet</code> was set</h3>
<p>This was due to actions being on the same level as the debug-output. Fixed</p>
<h3>Annoying warning at the start of the run</h3>
<p>For certain solvers the plot utilities issued a warning during a phase when no information about the current time is available. This is now fixed</p>
<h3>Redirected values</h3>
<p>not used when iterating over dictionaries When iterating over dictionaries with redirects the values in the redirected dictionaries were not used. This is now fixed</p>
<h3>Behavior of Template-engine not consistent in Python3 and Python2</h3>
<p>Due to a difference in the behavior of the <code>eval</code>-function in Python 3 certain expressions (especially with list comprehension) failed. Fixed</p>
<h3>Braces, brackets, parentheses in column name broke <code>RunDatabase</code></h3>
<p>These characters were stripped out by <em>SQLite</em>. They are thus normalized to special character combinations (which will be denormalized after reading)</p>
<h3>Finding of installations in alternate locations broken</h3>
<p>The algorithm to find (Open)FOAM-installations in alternate locations was broken. Now working again</p>
<h3>Failing on 3.x if socket for server thread already occupied</h3>
<p>Due to a a change in the socket API if the socket for the network thread (usually port 18000) was already notified the program failed. Fixed. Tested on 2.7 as well</p>
<h2>Enhancements to Utilities</h2>
<h3>=pyFoamPrepareCase= recognizes multi-region cases</h3>
<p>If there are multiple regions and no <code>prepareMesh.sh</code> then it will try to execute <code>blockMesh</code> for the regions</p>
<h3>=pyFoamPrepareCase= adds specialized templates</h3>
<p>With the option <code>--extension-addition</code> additional templates that override the regular templates can be specified.</p>
<p>Application may be for instance special templates for <code>potentialFoam</code></p>
<h3>=pyFoamPrepareCase= keeps data generated by meshing script</h3>
<p>If the meshing stage generates a <code>0</code>-directory then data in that directory will be kept. This can be switched off if this is not the desired behaviour</p>
<h3>=pyFoamPrepareCase= adds possibility for a file with default values</h3>
<p>If a file <code>default.parameters</code> is found in this directory then this file is automatically read. This file is read like a regular parameter-file but it can also have added information about the values:</p>
<ul>
<li><p>if an entry is a dictionary and has the entries <code>description</code> and <code>values</code> then it is assumed that this is a <em>group of values</em>. These groups are only used for grouping in reports. The <code>values</code> are set in the global scope.</p>
Groups can be nested</li>
<li>if an entry is a dictionary and has the entries <code>description</code> and <code>default</code> then it is assumed that the dictionary holds meta-information about the value. The entry <code>default</code> is the value that is actually used.</li>
<li>an additional enty <code>options</code> in that dictionary specifies a list of valid values for that parameter. The utility fails if the set value is not in that list. This is useful for setting boundary condition types or similar entries where only a limited number of words is valid</li>
</ul>
<p>It is recommended that <strong>all</strong> used values are specified in this file as this will be used for reporting</p>
<h3>=pyFoamPrepareCase= writes report about the variables</h3>
<p>The utility now writes a <em>Restructured Text</em> file with a report about the variables. Information from <code>default.parameters</code> like groupings and <code>description</code> are used in this report. Also the default value and the actual value are reported.</p>
<p>The file can be converted with a utility like <code>rst2pdf</code></p>
<h3>Gnuplot can be styled with default commands</h3>
<p>For all utilities that use Gnuplot as the backend for plotting there is now a configuration option <code>gnuplotCommands</code> in the <code>Plotting</code>-section with the usual <code>set</code> commands of Gnuplot. This is preset to display a grid and the y-axis.</p>
<p>The settings can be reset with the <code>gnuplotCommands</code>-list in the <code>customRegexp</code>-entries</p>
<h3>=pyFoamPVSnapshot.py= now supports Paraview 4.2 and later</h3>
<p>The API for screenshots changed with Paraview 4.2 and later. This API now supports layouts if multiple views were specified. The default behaviour is now to make only one screenshot per layout. The old behaviour (one screenshot per view) can now be switched on with the <code>-no-layouts</code>-option.</p>
<p>This allows screenshots exactly the way they look on screen</p>
<h3>=pyFoamPVSnapshot.py= allows switching between decomposed and reconstructed data</h3>
<p>The utility now rewrites the state file so that either decomposed or reconstructed data is read. The default is that the data set for which more timesteps exist is selected</p>
<h3>=pyFoamPVSnapshot.py= allows changing the field for sources</h3>
<p>It is now possible to specify a dictionary with source names and the fields that should be set for them. This allows quickly coloring the same layout with different fields.</p>
<p>This works for</p>
<ul>
<li>3D (rendered) filters</li>
<li>bar charts</li>
</ul>
<h3>=pyFoamPVSnapshot.py= allows rescaling the color-legend</h3>
<p>There are now two ways to rescale the color-transfer functions</p>
<ul>
<li>by specifying a dictionary with the ranges</li>
<li><p>by specifying a source. The range of the data on that source will be used to scale the data</p>
<ul>
<li>when specifying a source percentiles can be specified for the highest and lowest percent of the cells to be ignored</li>
</ul></li>
</ul>
<p>The first method will override the second</p>
<h3>=pyFoamPVsnapshot= reads parameters written by <code>pyFoamPrepareCase.py</code></h3>
<p>An option allows specifying that these parameters should be read. They are then available for substitution in the <em>Text</em> source</p>
<h3>=pyFoamListCases.py= allows filtering</h3>
<p>The utility now has options to select the cases that should be considered by name of the case. Either substrings or globs can be used. Ignore patterns can be specified as well</p>
<h3>=pyFoamRunParametervariation.py= now allows dictionaries</h3>
<p>The format of the parameter-file has been extended so that instead of values variations can be dictionaries. The contents of the dictionary are then used instead of a single value. An example would look like this</p>
<pre><code>defaults {
         a 1;
         b 2;
         c 3;
}

values {
       solver (
              interFoam
       );
       inVel ( 1 2 );
       outP ( 0 1 );
       sets (
            { name foo; a 3; }
            { name bar; b 4; c 5; }
       );
}
</code></pre>
<p>where sets has two variations. Values unset in <code>sets</code> will be used from <code>defaults</code></p>
<h3>=pyFoamConvertToCSV.py= now has all functionality of <code>pyFoamJoinCSV.py</code></h3>
<p>Now all functionality of the <code>Join</code>-utility is present in the <code>Convert</code> utility (including interpolating times)</p>
<p><code>pyFoamJoinCSV.py</code> will be removed in future versions of <code>PyFoam</code></p>
<h3>=dynamic= in <code>customRegexp</code> now allows composition from multiple match-groups</h3>
<p>If <code>idNr</code> is a list then now the actual id is composed from all the match groups specified in that list. If <code>idNr</code> is an integer then it behaves like it used to.</p>
<p>Application for the new behavior would be for instance to have the flow of different species on different patches in one plot</p>
<h3>New type <code>dynamicslave</code> in <code>customRegexp</code></h3>
<p>This combines the properties of the types <code>dynamic</code> and <code>slave</code>: dynamically generated data sets that are added to another plot</p>
<h3>Additional profiling option <code>--profile-line-profiler</code></h3>
<p>Uses the library <code>line_profiler</code> for profiling. Only of interest for developers. Experimental</p>
<h3>Utilities that use templates can be customized with the configuration</h3>
<p>The section <code>Template</code> in the configuration now allows modifying the behavior of the templating engine (how templates are processed and syntax details)</p>
<h3>=LocalConfigPyFoam= now can be read <strong>before</strong> argument parsing</h3>
<p>This allows overruling options before they are set be command line options. This has to be enabled by the application (it doesn't make sense for all).</p>
<p>An example is the <code>pyFoamPrepareCase.py</code>-utility where the local configuration file can overrule the default behavior of the template engine</p>
<h3>=pyFoamConvertToCSV.py= automatically selects the output format with <code>--automatic-format</code></h3>
<p>Now if that option is selected and the extension of the output is <code>xls</code> the option <code>-write-excel-file</code> doesn't have to be set anymore</p>
<h3>=pyFoamConvertToCSV.py= allows adding original data as separate sheets</h3>
<p>The input data files now can be added to an excel-file as separate sheets with the <code>--add-sheets</code>-option</p>
<h3>=pyFoamConvertToCSV.py= has improved naming of columns</h3>
<p>Now if there is more than one source file then the columns from the first source also get a unique prefix.</p>
<p>It is also possible to clean the column names before writing them to file. The time name can be changed with <code>-write-time-name</code>. Substrings can be replaced with <code>--column-name-replace</code> and simple functions can be applied with <code>--column-name-transformation</code></p>
<h3>=pyFoamConvertToCSV.py= now supports sets-files</h3>
<p>The utility now can read these files and determine the field names from the filename. <code>--automatic</code> assumes that files with the extension <code>.xy</code> are of this format</p>
<h3>=pyFoamPrepareCase.py= can calculate derived values with a script</h3>
<p>If a script <code>derivedParameters.py</code> is present then it is executed and values set in that script can be used in the templates as well</p>
<h3>=pyFoamPrepareCase.py= adds a variable <code>numberOfProcessors</code></h3>
<p>If unspecified by the user this variable is automatically <span class="math">1</span>. It can be used by the templates to distinguish between different cases</p>
<p>The <code>PrepareCaseJob</code>-class in <code>ClusterJob</code> automatically sets the values according to the number of processors in the cluster job</p>
<h3>=pyFoamChangeBoundaryName.py= and <code>pyFoamChangeBoundaryType.py</code> now support decomposed cases</h3>
<p>Both utilities now also modify the boundary files in the <code>processorX</code>-direcories. This behaviour can be switched off from the command line</p>
<h3>=pyFoamPrepareCase.py= has possibility for templates after the final stage</h3>
<p>Templates with the extension <code>.finalTemplate</code> are executed after the <code>caseSetup.sh</code>-script.</p>
<h3>=pyFoamRunParameterVariation= allows adding postfix to cloned cases</h3>
<p>The option <code>--cloned-case-postfix</code> adds a postfix to the cloned directory names. This can be used if the results from multiple variations with the same parameter file should be kept (for instance when comparing OpenFOAM-versions)</p>
<h3>=pyFoamConvertToCSV= now allows setting of default input file format</h3>
<p>The option <code>--default-read-format</code> now allows setting a different format than <code>csv</code> as the default format for input files</p>
<h3>=pyFoamListCases.py= adds the hostname to the printed information</h3>
<p>The utility tries to determine from the pickled data the host the simulation was run on and prints it (can be switched off with an option)</p>
<h3>=pyFoamPrepareCase.py= allows cloning</h3>
<p>The utility now has an option <code>--clone-case</code> to clone a new case before starting instead of working in the specified directory (in this case the case will be cloned to this directory). The name of the created directory <strong>can</strong> be constructed from the specified parameters with the <code>--automatic-casename</code>-option</p>
<h2>Enhancements to the Library</h2>
<h3>=SolutionDirectory= detects multiple regions</h3>
<p>Valid regions are sub-directories of <code>constant</code> that have a <code>polyMesh</code>-directory</p>
<h3>=BoolProxy= now compares like builtin <code>bool</code></h3>
<p>Comparison used to fail for types where it was not explicitly implemented like <code>None</code></p>
<h3>=PyFoamApplication=-class now supports <code>pvpython</code> for debugging</h3>
<p>Now <code>--interactive-after-exception</code> also works in the utilities that use <code>pvpython</code></p>
<h3>=TemplateFile= now allows more flexible assignments</h3>
<p>In the mode where executions are allowed now more flexible assignments to variables are possible. To be specific:</p>
<ul>
<li>array assignments like</li>
</ul>
<pre class="example"><code>$$ a[&quot;t&quot;] = 2
</code></pre>
<ul>
<li>multiple assignments like</li>
</ul>
<pre class="example"><code>$$ a,b = 2,3
</code></pre>
<h3>=ThirdParty=-library <code>six</code> upgraded to 1.9.0</h3>
<p>This library has been upgraded to the latest released version</p>
<h3>Additional markup in <code>RestructuredTextHelper</code></h3>
<p>There are additional methods <code>emphasis</code>, <code>strong</code> and <code>literal</code> that wrap their arguments in the appropriate markup</p>
<p>The methods <code>bulletList</code>, <code>enumerateList</code> and <code>definitionList</code> take lists or dictionaries and mark them as lists</p>
<h3>=SpreadsheetData= can now read files produced by the <code>sets</code>-functionObject</h3>
<p>If the option <code>isSampleFile</code> is set then it is assumed that the field names are in the filename and there is no header with field names in the file</p>
<h2>Infrastructure</h2>
<h3>Adaption of Debian packaging to new conventions</h3>
<p>By Oliver Borm. The building of Debian packages for Python libraries has changes. Necessary adaptions were done by Oliver Borm</p>
<h2>Development changes</h2>
<h3>Now uses <code>pytest</code> for unittesting</h3>
<p>The <code>nose</code>-library had problems and all the unit-tests run out-of-the-box with <code>pytest</code></p>
<h1>Version 0.6.4 - 2014-11-24</h1>
<h2>Requirements</h2>
<p>The only requirement is a python-installation. The main testing and development has been done with 2.7 and 2.6. Python 2.5 should work but not for all utilities. Currently 2.6, 2.7 and 3.4 are used during development.</p>
<p>Certain functionalities need special Python-libraries. The main one is <code>numpy</code> for plotting. Other libraries that might be of interest are <code>matplotlib</code>, <code>pandas</code> and <code>IPython</code>. For <code>pyFoamDisplayBlockmesh.py</code> the libraries <code>PyQt4</code> and <code>vtk</code> are needed. Other libraries are tested for and reported by <code>pyFoamVersion.py</code> but not required (only install if you're sure that you need it)</p>
<h2>Future changes</h2>
<h3>Redundant utilities <code>pyFoamJoinCSV.py</code> and <code>pyFoamConvertToCSV.py</code> unified</h3>
<p>These two utilities are almost indistinguishable and will be unified into one</p>
<h2>Major changes</h2>
<h3>Multi-line regular expressions in <code>customRegexp</code></h3>
<p>If in <code>customRegexp</code> an <code>expr</code> is found with <code>\n</code> then this expression is matched over multiple consecutive lines. Types like <code>dynamic</code> work as usual.</p>
<p>This makes it possible to match for instance the output of the <code>forces</code>-function objects</p>
<h3>Enhancement of <code>pyFoamPrepare.py</code></h3>
<p>The utility which was introduced in the last version is becomong more usable and will be central to all things that set up the case (for instance a special <code>ClusterJob</code>)</p>
<h3>Enhancements of the CSV-utilities</h3>
<p>These utilities are now more flexible and allow writing and reading of Excel-files too</p>
<h3>Environment variable <code>PYFOAM_SITE_DIR</code> and <code>PYFOAM_DIR</code></h3>
<p>Both variables are not necessary but <code>PYFOAM_SITE_DIR</code> allows consistent insertion of site-specific libraries and utilities.</p>
<p><code>PYFOAM_DIR</code> is set by some Foam-distributions and tested for by <code>pyFoamVersion.py</code>. It is supposed to point to the currents PyFoam-installation.</p>
<p><code>PYFOAM_SITE_DIR</code> points to a directory with site-specific scripts and configurations. The sub-directories supported (and thested py <code>pyFoamVersion.py</code>) are</p>
<dl>
<dt>bin</dt>
<dd>directory with scripts. It has to be added to the <code>PATH</code> outside of PyFoam (for instance in the <code>.bashrc</code>)
</dd>
<dt>etc</dt>
<dd>Optional directory which is searched for configuration files. Priority is below the user settings and above the global settings
</dd>
<dt>lib</dt>
<dd>Optional directory to allow mixing in site-specific library files. Imported as <code>PyFoam.Site</code>: For instance if a file <code>Foo.py</code> is present in <code>lib</code> it can be imported as <code>PyFoam.Site.Foo</code>. This directory does not have to be (in fact: it <strong>shouldn't</strong>) added to <code>PYTHONPATH</code>
</dd>
</dl>
<p>Purpose of <code>PYFOAM_SITE_DIR</code> is to allow administrators to provide site-wide scripts and settings for all users on a site</p>
<h2>Incompatibilities</h2>
<h3>Option <code>--silent</code> removed from <code>pyFoamPrepareCase.py</code></h3>
<p>Option has been renamed to <code>--no-complain</code></p>
<h3>Keys in <code>RunDatabase</code> with column-names that contain upper-case letters change</h3>
<p>SQLite does not support case-sensitive column-names (<code>s_max</code> and <code>S_max</code> are the same). To change this the upper case letters in the column names are replaced by an underscore and the letter (<code>S_max</code> becomes <code>_s__max</code>)</p>
<p>This means that old databases might not be read correctly</p>
<h3>Change in unique variable names in <code>pyFoamConvertToCSV.py</code></h3>
<p>The algorithm to make variable names unique has changed (basically it uses the part of the filenames that differ) and scripts relying on these names might fail</p>
<h3>=PyFoam.IPython=-module renamed to <code>PyFoam.IPythonHelpers</code></h3>
<p>The name of the module crashed in certain instances (especially unit-testing) with the regular <code>IPython</code>-library. To avoid these crashes it has been renamed to <code>IPythonHelpers</code>. This raises two potential problems:</p>
<ul>
<li>scripts that <code>import</code> the module have to be adapted to the new name</li>
<li>IPython-notebooks created with <code>pyFoamIPythonNotebook.py</code> have two imports pointing to this module. These notebooks have to be adapted to be usable again</li>
</ul>
<h2>Bugfixes</h2>
<h3>Templates in <code>pyFoamPrepareCase.py</code> did not keep permissions</h3>
<p>This was a problem for script-templates which were not executable any more. Fixed</p>
<h3>=pyFoamComparator.py= failed due to circular dependency</h3>
<p>This has been fixed by adding an import in <code>BasicRunner.py</code></p>
<h3>=pyFoamDumpRunDatabaseToCSV.py= fails if Pandas-data is requested</h3>
<p>This is now fixed</p>
<h3>=sort= for list broke code on Python 3</h3>
<p>Some calls for <code>sort</code> still used the <code>cmp</code>-parameter which does not exist for Python3 anymore. These calls have been replaced with <code>key</code> and <code>reverse</code></p>
<h3>Changing the OF-version does not work in Python 3</h3>
<p>Because the output of <code>subprocess</code> is now <em>binary</em> instead of a regular string. Fixed</p>
<h3>=addData= in <code>PyFoamDataFrame</code> extrapolates for invalid values</h3>
<p>This was due to incorrect use of the <code>interpolate</code>-method</p>
<h3>=--keep-last= did not work for <code>pyFoamClearCase.py</code> and parallel cases</h3>
<p>This was because there was a problem in the library code and the utility did not consider the parallel time-steps. Fixed</p>
<h3>=pyFoamDumpRunDatabaseToCSV.py= does not add basic run information</h3>
<p>Basic run information was not added to the file. Now it is with the prefix <code>runInfo//</code></p>
<h3>Restore of <code>FileBasisBackup</code> did not work</h3>
<p>The logic for checking whether a file was &quot;backupable&quot; was wrong. This affected the proper restore of files with utilities for instance for <code>--write-all</code></p>
<h3>Remove circular dependency in <code>DataStructures</code></h3>
<p>According to the bug <a href="http://sourceforge.net/p/openfoam-extend/ticketspyfoam/219/">http://sourceforge.net/p/openfoam-extend/ticketspyfoam/219/</a> it was not possible to import <code>DataStructures</code> because of a circular dependency with <code>FoamFileGenerator</code>. Fixed by moving an import to the back of the file</p>
<h2>New features/Utilities</h2>
<h3>=pyFoamRunParameterVariation.py=</h3>
<p>This utility takes a template case and a file specifying the parameter variation and creates cases with the <code>pyFoamPrepareCase.py</code>-engine, runs a solver on these cases and collects the data into a database. The database can then be extracted with <code>pyFoamDumpRunDatabaseToCSV.py</code></p>
<h3>=pyFoamBinarySize.py=</h3>
<p>Calculates the size of the binaries in an OpenFOAM-installation separated by compile-option</p>
<h3>=pyFoamBlockMeshRewrite.py=</h3>
<p>Assists the user in rewriting the <code>blockMeshDict</code> by doing simple, but error-prone transformations. Assumes &quot;sensible&quot; formatting: one block/vertex etc per line.</p>
<p>Sub-commands are:</p>
<dl>
<dt>refine</dt>
<dd>refines mesh by multiplying cell numbers in the blocks
</dd>
<dt>number</dt>
<dd>Adds comments with the vertex numbers. Should help the user when editing/modifying the mesh
</dd>
<dt>stripNumber</dt>
<dd>Remove the comments added by <code>number</code>
</dd>
<dt>mergeVertices</dt>
<dd>Adds vertices from other blockMeshes that are not present in the current blockMesh
</dd>
<dt>renumberVertices</dt>
<dd>Take another <code>blockMeshDict</code>, copy over the <code>vertices</code>-section of that mesh and rewrite <code>blocks</code> and <code>patches</code> so that they conform to these <code>vertices</code>. The original <code>vertices</code> have to be a sub-set of the <code>vertices</code> in the other mesh
</dd>
<dt>normalizePatches</dt>
<dd>Rotates patches so that the lowest number is in front
</dd>
</dl>
<h2>Enhancements to Utilities</h2>
<h3>=pyFoamChangeBoundaryType.py= allows setting additional values</h3>
<p>The option <code>--additional-values</code> allows specifying a dictionary with additional values for the boundary (stuff that is needed by <code>mappedWall</code> etc)</p>
<h3>=pyFoamPrepareCase.py= now has OF-version and fork as defined variables</h3>
<p>This allows to write case-templates that can distinguish between different OF-versions</p>
<h3>=pyFoamPrepareCase.py= now allows &quot;overloading&quot; another directory</h3>
<p>Before doing anything else the contents of different directories are copied into the current case. This allows for instance to use tutorial cases as the basis for a case</p>
<h3>=pyFoamIPythonNotebook.py= adds improvements to the notebook</h3>
<p>Additional code added to the generated notebook:</p>
<ul>
<li>Code to change the default size of the plots</li>
<li>Distribution-directories in subdirectories <code>distributions</code> (generated by some <code>swak</code>-function objects) added</li>
</ul>
<h3>=pyFoamListCases.py= more tolerant to faulty <code>controlDict</code></h3>
<p>If the <code>controlDict</code> is acceptable to OpenFOAM but syntactically incorrect for PyFoam (for instance because of a missing semicolon) the utility does not fail anymore (but no data is collected for that case).</p>
<h3>=pyFoamDumpConfiguration.py= prints sections and keys alphabetically</h3>
<p>This should make it easier to find items</p>
<h3>=pyFoamJoinCSV.py= and <code>pyFoamConvertToCSV.py</code> read and write Excel-files</h3>
<p>Both utilities now allow writing Excel-files</p>
<p>In addition to regular text files the first sheet from <code>xls</code>-files can be read</p>
<h3>Flexible variable filtering in <code>pyFoamJoinCSV.py</code> and <code>pyFoamConvertToCSV.py</code></h3>
<p>Now it is possible to filter for regular expressions</p>
<p>The functionality of the two utilities now is very similar and it is possible that one of them will be discontinued</p>
<h3>Columns in <code>pyFoamJoinCSV.py</code> and <code>pyFoamConvertToCSV.py</code> can be recalculated</h3>
<p>The two utilities now can add columns or recalculate columns based on the existing column values</p>
<h3>Testing for <code>Numeric</code> removed from <code>pyFoamVersion.py</code></h3>
<p>Testing for the library <code>Numeric</code> library removed as it is no longer supported as a fallback for <code>numpy</code>. Test also removed from <code>setup.py</code></p>
<h2>Enhancements to the Library</h2>
<h3>Subclass of <code>ClusterJob</code> that support <code>PrepareCase</code></h3>
<p>The class <code>PrepareCaseJob</code> supports cases that are set up with <code>pyFoamPrepareCase.py</code>. Additional parameters to the constructor are</p>
<ul>
<li>the name of the parameter-file</li>
<li>a list with the parameters. The list is composed of name/value-pairs</li>
</ul>
<h3>Subclass of <code>ClusterJob</code> that support <code>RunParameterVariation</code></h3>
<p>The class <code>VariationCaseJob</code> supports cases that are set up with <code>pyFoamRunParameterVariation.py</code>. Additional parameters to the constructor are</p>
<ul>
<li>the name of the parameter-file</li>
<li>the name of the variations-file</li>
</ul>
<h3>=execute= in <code>PyFoam/Utilities</code> fails if script is not executable</h3>
<p>The function checks if the file exists and is <strong>not</strong> executable. The program fails in that case</p>
<h3>=foamVersion= uses a separate wrapper class for <code>tuple</code></h3>
<p>This ensures that it is printed in a form that is valid in OF-dictionaries</p>
<h3>Move calculation of disk usage to <code>Utilities</code></h3>
<p>This has until now only been used in <code>ListCases</code> but moved to a separate method/function <code>diskUsage</code> in the <code>Utilities</code>-module</p>
<h3>Enhancement of <code>--help</code></h3>
<p>Added the possibility to have an epilog and usage examples with the <code>epilog</code> and <code>examples</code>-keyword arguments for applications.</p>
<p>These and descriptions now have the possibility for line-breaks: if two line-breaks are encountered in the text a new paragraph is created</p>
<h3>=which=-routine in <code>Utitlities</code> uses native Python-routine</h3>
<p>For Python-version where <code>shutil</code> has a <code>which</code>-function this is used instead of calling an external program</p>
<h3>=FileBasis= now allows file handles instead of the filename</h3>
<p>This currently only works for reading, Backups, zipping etc won't work but it makes algorithms more flexible</p>
<h3>=BlockMesh= doesn't force writing to file anymore</h3>
<p>Instead content is stored in memory. Old behaviour is the default to preserve compatibility with old scripts</p>
<h3>Additional methods for <code>BlockMesh</code>-class</h3>
<dl>
<dt>numberVertices</dt>
<dd>Adds comments with the vertex numbers to the vertices
</dd>
</dl>
<h3>=LineReader= allows keeping spaces on left</h3>
<p>Previous behaviour was stripping all spaces from the lines. Now the left hand spaces can be ket. Old behaviour is still default for compatibility</p>
<h3>=TemplateFile= now allows writing of assignment-results in file</h3>
<p>This allows faster debugging of template-files. This can be enabled with a switch in the utilities using templates</p>
<h3>=SolverJob= now allows passing of parameters to the solver</h3>
<p>And additional parameter <code>solverArgs</code> will now be passed to the solver (if the solver accepts arguments)</p>
<h3>=SpreadsheetData= now allows reading from an Excel file</h3>
<p>During construction if an Excel-file is specified and the <code>xlrd</code>-library and <code>pandas</code> are installed then the first sheet in the file is read</p>
<h3>=SpreadsheetData= allows recalculating columns</h3>
<p>Columns can be recalculated using expressions. This includes other data items. Currently present column names are available as variables. There is also a variable <code>data</code> that can be subscripted for items that are not valid variable names. A variable <code>this</code> points to the item to be recalculated</p>
<h2>Known bugs</h2>
<h3>Timelines not forgotten for multiple runner calls</h3>
<p>This manifests with <code>pyFoamRunParameterVariation.py</code>. The custom timelines are still kept in memory. Not a problem. Just annoying</p>
<h1>Version 0.6.3 - 2014-06-23</h1>
<h2>Requirements</h2>
<p>The only requirement is a python-installation. The main testing and development has been done with 2.7 and 2.6. Python 2.5 should work but not for all utilities. Unit tests run on Python 3.4 but it is currently not used in a production environment (reports of success or failure most welcome)</p>
<p>Certain functionalities need special Python-libraries. The main one is <code>numpy</code> for plotting. Other libraries that might be of interest are <code>matplotlib</code>, <code>pandas</code> and <code>IPython</code>. For <code>pyFoamDisplayBlockmesh.py</code> the libraries <code>PyQt4</code> and <code>vtk</code> are needed. Other libraries are tested for and reported by <code>pyFoamVersion.py</code> but not required (only install if you're sure that you need it)</p>
<h2>Major changes</h2>
<h3>Version changing supports forks of OpenFOAM</h3>
<p>Now <code>pyFoam</code> supports different versions of OpenFOAM for switching. Out of the box <code>openfoam</code> and <code>extend</code> are supported. If only the version number is specified (for instance by <code>--foamVersion=1.7.x</code>) and such a version exists only for one fork it is correctly expanded with the correct fork( in the example with <code>openfoam-1.7.x</code>). If more than one fork has the same version then the fork name has to be specified as well</p>
<p>Note: additional forks can be easily specified with the configurations. In section <code>OpenFOAM</code> the parameter <code>forks</code> has to be extended. For each new fork a <code>dirpatterns</code> and <code>installation</code>-parameter has to be specified</p>
<h2>Incompatibilities</h2>
<h3>Change of command interface of <code>pyFoamSTLUtility.py</code></h3>
<p>The selection of what is to be done is now selected by subcommands instead of options. This will break scripts using this</p>
<h3>If <code>0.org</code> is present <code>pyFoamCloneCase.py</code> and <code>pyFoamPackCase.py</code> ignore <code>0</code></h3>
<p>The reason is that the utilities assume that this directory is produced from <code>0.org</code></p>
<h2>Bugfixes</h2>
<h3>PlotWatcher has long times between updates if pickling takes long</h3>
<p>The reason was that it used the same throttling that made sense for the PlotRunner. Fixed</p>
<h3>=pyFoamPVSnapshot.py= fails for newer paraview-versions</h3>
<p>Reason is that the class <code>vtkPythonStdStreamCaptureHelper</code> does not support <code>isatty</code></p>
<h3>SamplePlot failed when valueNames are unspecified</h3>
<p>Reported in <a href="https://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=208">https://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=208</a> and fixed</p>
<h3>=pyFoamTimelinePlot.py= failed Numpy/Pandas output of vector fields</h3>
<p>Vector fields only were added to the data fields if they were the first in the list. Fixed</p>
<h3>=alternateAxis= ignored for slave</h3>
<p>This is now fixed. The alternate values have to be specified in the master (specifying in the slave gives an error)</p>
<h3>=pyFoamCaseReport.py= more stable for binary <code>boundary</code>-files</h3>
<p>Usually these files are <code>ascii</code> (even if the header says <code>binary</code>). In some cases the parsing failed for these. Fixed by enforcing reading as <code>ascii</code>. Can be switched off</p>
<h3>=SpreadsheetData= returns data which breaks certain Pandas-operations</h3>
<p>The reason was that if there were duplicate times in the table the index was non-unique which certain Pandas-operations don't appreciate. Solved by dropping duplicate times. Can be switched off</p>
<h3>=pyFoamCloneCase.py= added duplicates to the archive</h3>
<p>If things are specified twice they were added twice. Now it is checked whether the item already exists in the tar-file before adding them</p>
<h3>=nonuniform= of length 3 not correctly printed</h3>
<p>The reason was that this was interpreted as a vector and the numeric prefix was removed. Reported at <a href="http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=218">http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=218</a></p>
<p>Fixed by introducing an extra parameter to <code>FoamFileGenerator</code></p>
<h2>New features/Utilities</h2>
<h3>=pyFoamPrepareCase.py= for case preparation</h3>
<p>This utility aims to reduce the need for boilerplate scripts to set up cases. The steps it executes are</p>
<ol>
<li>Clear old data from the case (including processor directories)</li>
<li>if a folder <code>0.org</code> is present remove the <code>0</code> folder too</li>
<li>go through all folders and for every found file with the extension <code>.template</code> do template expansion using the pyratemp-engine</li>
<li>create a mesh (either by using a script or if a <code>blockMeshDict</code> is present by running blockMesh. If none of these is present assume that there is a valid mesh present)</li>
<li>copy every <code>foo.org</code> that is found to to <code>foo</code> (recursively if directory)</li>
<li>do template replacement for every <code>.postTemplate</code></li>
<li>execute another preparation script</li>
</ol>
<h3>=pyFoamIPythonNotebook.py= for generating and manipulating IPython-notebooks</h3>
<p>This utility creates and manipulates IPython-notebooks for analyzing OpenFOAM cases. It has a number of subcommands:</p>
<dl>
<dt>create</dt>
<dd><p>this is the main command. All other commands assume that the notebooks they work with were created with this.</p>
<p>The utility looks at the case specified and creates a notebook that has the capabilities to quickly build a report about the case:</p>
<ul>
<li>reporting general properties of the case. This</li>
</ul>
</dd>
</dl>
<p>basically is the capability of the <code>pyFoamCaseReport.py</code>-utility</p>
<ul>
<li>It searches for data that can be visualized by <code>pyFoamTimelinePlot.py</code> or <code>pyFoamSamplePlot.py</code> and generates selectors that allow the user to select which data to import. The selectors import the data as Pandas-=DataFrames= and create the commands necessary to do this. It is recommended to erase the selector afterwards</li>
<li>Selectors for pickled case data and pickled plot generated by PyFoam</li>
<li>Capability to store read data <strong>in</strong> the notebook file. This feature is experimental and has performance issues for medium to large datasets</li>
</ul>
<p>The created notebook can be executed but needs to be edited to be useful</p>
<dl>
<dt>clear</dt>
<dd>removes selected cells (but only cells created with <code>create</code>) or output from the notebook.
</dd>
<dt>info</dt>
<dd>prints information about the notebook
</dd>
<dt>copy</dt>
<dd>copies notebook to a different case and rewrites it so that data is read from that case
</dd>
</dl>
<p>Recommended way of working with this utility is</p>
<ol>
<li>Create notebook with utility</li>
<li>Edit it to contain standardized evaluations</li>
<li>Copy it over to another, similar case</li>
</ol>
<h3>Additional sub-module <code>PyFoam.IPython</code></h3>
<p>The purpose of this submodule is to support <code>pyFoamIPythonNotebook.py</code>. It has the classes</p>
<dl>
<dt>Notebook</dt>
<dd>read a file and interpret it as an IPython-notebook. Do manipulations on this notebook
</dd>
<dt>PermanentStorage</dt>
<dd>Implements permanent storage in an IPython-notebook. Only works inside a notebook and allows only one instance at once. Passing the data from the notebook (through JavaScript) to Python currently is a performance bottleneck
</dd>
<dt>Case</dt>
<dd>Convenience object that exposes the functionality of some of the PyFoam-utilities through a simple interface
</dd>
</dl>
<h3>Additional sub-module <code>PyFoam.Wrappers</code></h3>
<p>Wraps popular Python-libraries to add functions that make it easier to work with OpenFOAM-data.</p>
<p>Currently only one Wrapper is implemented:</p>
<h4>=Pandas=-wrappers</h4>
<p>This provides <code>PyFoamDataFrame</code> as a wrapper for <code>DataFrame</code>. The functionality added is</p>
<dl>
<dt>addData</dt>
<dd>Conveniently add new data from different <code>Series</code>, <code>DataFrames</code>. It is assumed that the index is the same property (time or for samples the distance) but with a different resolution. The indexes are joined and missing data is interpolated
</dd>
<dt>integrate,</dt>
<dd>uses the index as the <span class="math"><em>x</em></span>-axis and calculates these properties for a <code>Series</code>. <code>validLength</code> is the extent on which data is defined (<code>weightedAverage</code> is basically <code>integrate</code> divided by <code>validLength</code>). For <code>integrate</code> the trapezoid-rule is used
</dd>
<dt>describe</dt>
<dd>adds the three above quantities to the regular <code>describe</code>-command
</dd>
</dl>
<h2>Enhancements to Utilities</h2>
<h3>=pyFoamSampleplot= has option to use index instead of time in filenames</h3>
<p>The option <code>-index-instead-of-filename</code> switches this on. This makes it easier to generate movies from the files</p>
<h3>=pyFoamListCases.py= Allows addition of custom data</h3>
<p>The option <code>--custom-data</code> now allows the specification of custom data items. These are read from the <code>pickledData</code>-files and displayed in the table like regular data items</p>
<h3>Switch compiler versions</h3>
<p>Now all utilities allow switching the compiler version (for instance from <code>Gcc47</code> to <code>Gcc48</code>). The relevant options are <code>--force-system-compiler</code>, <code>--force-openfoam-compiler</code> and <code>--force-compiler</code></p>
<h3>=pyFoamVersion.py= reports the installed versions better</h3>
<p>Now the location of the installations is reported as well</p>
<h3>Offscreen rendering can be switched off in <code>pyFoamPVSnapshot.py</code></h3>
<p>This is a workaround where the writer produces a segmentation fault</p>
<h3>Write 3D-data in <code>pyFoamPVSnapshot.py</code></h3>
<p>In addition to writing out bitmaps allows writing out 3D-data (for importing into other applications). Sources can be selected by name</p>
<h3>Added capabilities to <code>pyFoamSTLUtility</code></h3>
<p>The utility can now also:</p>
<ul>
<li>erase selected patches</li>
<li>merge selected patches into one</li>
</ul>
<h3>=pyFoamDecomposer.py= switches off function objects</h3>
<p>This now automatically happens for OF-versions that support it (2.0 and greater). They can be switched on again</p>
<h3>=pyFoamCloneCase.py= clones more stuff</h3>
<p>Files that are assumed to be used by <code>pyFoamPrepareCase.py</code> are automatically added to the clone. This includes all files (and directories) with the extensions <code>.sh</code>, <code>.template</code> and <code>.org</code>. Also IPython notebooks (extension <code>.ipynb</code> are added)</p>
<h2>Enhancements to the Library</h2>
<h3>=BasicRunner= now can print the command line that is actually used</h3>
<p>This should help with diagnosing problems with MPI etc.</p>
<p>Can be switched on in some utilities with <code>--echo-command-prefix</code></p>
<h3>=ClusterJob= now can live without a machinefile</h3>
<p>Using the machine-file now can be switched off for job-schedulers with a tight integration</p>
<h3>Enhanced treatment of symlinks during cloning</h3>
<p>If a item in the case itself is a symlink then it used to be a copy of the file the symlink is pointing to. Now it is created as a symlink to the target the original symlink. If the <code>--follow-symlink</code>-option is used the old behaviour is used (copying). In this case the option <code>noForceSymlink</code> in the <code>Cloning</code>-section of the configuration can be used to change this behaviour for selected files</p>
<h3>=AnalyzedCommon= clears the <code>analyzed</code>-directory</h3>
<p>The directory is cleared if it exits from a previous run.</p>
<h3>=TimelineDirectory= is more tolerant</h3>
<p>Used to fail if incompatible data types were used. Now ignores them</p>
<h3>Possibility of a subcommand-interface for utilities</h3>
<p>The subclass <code>SubclassFoamOptionParser</code> now allows the parsing of subclasses. The base class for utilities <code>PyFoamApplication</code> now supports this as an option. As an example this is implemented in <code>pyFoamSTLUtilities.py</code></p>
<h3>=STLUtility= accepts file-handles</h3>
<p>The class checks whether arguments are filehandles and in this case doesn't try to open a file for reading or writing but uses the handle</p>
<h3>=addClone= in <code>SolutionDirectory</code> accepts glob patterns</h3>
<p>If no file matching the name is found it is assumed that this is a glob-pattern and all matching files are added. This affects all utilities that use that method (especially <code>pyFoamCloneCase.py</code>)</p>
<h3>=execute= in <code>Utilities</code> allows specification of working directory and echoing of output</h3>
<p>This method now allows the specification of a working directory. Before executing the command the method changes to the working directory. Afterwards it changes back to the regular working directory.</p>
<p>There is also an option <code>echo</code> that immediately prints the output to the screen</p>
<h3>=rmtree= and <code>copytree</code> more tolerant</h3>
<p><code>rmtree</code> now also works if the &quot;tree&quot; is a file.</p>
<p><code>copytree</code> now has a parameter <code>force</code> that allows removing the destination directory if it exists</p>
<h3>Enhanced support for booleans in the parser</h3>
<p>Strings that are usually interpreted as boolean in OF-dictionaries (for instance <code>on</code>, <code>yes</code>, ...) are now stored as a special type that allows treating them like 'real' booleans.</p>
<p>For instance an expression <code>test no;</code> in a dictionary now allows things like <code>if d['test']:</code> in the script</p>
<h3>Application classes now allow specifying options as keyword parameters</h3>
<p>Until now the options to be used had to be specified as a list of string modeled on the way the command line looked like. This is still possible. In addition it is now possible to specify these things as keyword parameters on the command line. Rudimentary type checking is done. The names of the parameters are generated from the command line options: the <code>-</code> are removed and the words are converted to CamelCase. So for instance <code>--list-custom-Regexp </code> becomes <code>listCustomRegexp</code>. Also for switches like these a boolean value has to be passed. So the correct usage in a call would be .</p>
<h3>=SolutionDirector= now can classify directories in the <code>postProcessing</code>-directory</h3>
<p>A number of properties has been added that list data generated by function objects:</p>
<dl>
<dt>timelines</dt>
<dd>timeline data (like <code>propes</code>) that can be visualized by <code>pyFoamTimelinePlot.py</code>
</dd>
<dt>samples</dt>
<dd>data from <code>set</code> (assuming it is in <code>raw</code>-format) that can be processed by <code>pyFoamSamplePlot.py</code>
</dd>
<dt>surfaces</dt>
<dd>data from <code>surface</code> (assumes <code>VTK</code>-format) that can be used by <code>pyFoamSurfacePlot.py</code>
</dd>
<dt>distributions</dt>
<dd>special cases of <code>sample</code> with distribution data
</dd>
</dl>
<p>These properties only list the subdirectories of the case with that data</p>
<p>Additional properties are</p>
<dl>
<dt>pickledData</dt>
<dd>a list of pickled data files that are found
</dd>
<dt>pickledPlots</dt>
<dd>list of found pickled plots
</dd>
</dl>
<p>These lists are sorted in descending temporal order (newest first)</p>
<h3>=pyFoamSamplePlot.py= now more flexible for distributions</h3>
<p>Tries to determine the names of the values from the first line in the files</p>
<h3>=DictProxy= now has a <code>dict</code>-like <code>update</code>-method</h3>
<p>This also allows enforcing string values</p>
<h3>=FoamFileGenerator= automatically quotes strings</h3>
<p>If strings are unquoted but contain characters that make it illegal as a word then the string is quoted before output</p>
<h3>Children of <code>FileBasis</code> now can be used with the <code>with</code>-statement</h3>
<p>This mainly concerns <code>ParsedParameterFile</code></p>
<h1>Version 0.6.2 - 2013-11-03</h1>
<h2>Major changes</h2>
<h3>Use of <code>pandas</code>-library</h3>
<p>Starting with this version the <code>pandas</code>-library is used for data-analysis. When possible and appropriate classes return <code>pandas</code>-objects. Currently these are:</p>
<ul>
<li><code>CSVCollection</code>. With a call-operator this class returns the collected data as a <code>DataFrame</code> of the collected data</li>
<li><code>SpreadsheetData</code> now has methods to return <code>Series</code> and <code>DataFrame</code> objects</li>
</ul>
<p>It is not necessary to install <code>pandas</code> if these classes are not used (and even then most of their functionality works)</p>
<h2>Incompatibilities</h2>
<h3>Different separator for databases in CSV-files</h3>
<p>The class <code>RunDatabase</code> (and therefor also the utility <code>pyFoamDumpRunDatabaseToCSV.py</code>) now write as a separator for data from sub-tables a <code>//</code> instead of the space. This especially means that scripts that rely on a data-item <code>foo</code> in <code>analyzed</code> might break because this is now called <code>analyzed//foo</code> instead of <code>analyzed foo</code>. On the other hand this makes the names more consistent and easier to parse as <code>//</code> is the saperator for other levels of dictionaries</p>
<h3>Change of independent variable name in sample data</h3>
<p>Instead of <code>col0</code> this is now <code>coord</code>. This could cause problems with scripts that use that column name in the resulting <code>SpreadsheetData</code>-object</p>
<h2>Bugfixes</h2>
<h3>=pyFoamPackCase.py= does not handle symbolic links correctly</h3>
<p>Symbolic links were copied as is and did not work correctly afterwards. This is fixed. If the symbolic link is an absolute path or points outside the case directory it is replaced with the file it points to. Otherwise it is preserved as a symbolic link</p>
<h3>=pyFoamPotentialRunner.py= not working with OpenFOAM 2.0 or newer</h3>
<p>These versions require an entry <code>potentialFlow</code> in the <code>fvSolution</code>-file instead of the old <code>SIMPLE</code></p>
<h3>=pyFoamListCase.py= fails with <code>controlDict</code> that use preprocessing</h3>
<p>Fixed by first trying to read that with preprocessing. Without if that fails</p>
<h3>Cloning fails in symlink-mode if files are specified twice</h3>
<p>Now using a <code>set</code> instead of a <code>list</code> makes sure that no file is cloned twice</p>
<h2>Utilities</h2>
<h3>=pyFoamPotentialRunner.py= now allows removing of <code>functions</code> and <code>libs</code></h3>
<p>The utility now allows removing these entries in case that they don't work with <code>potentialFoam</code></p>
<h3>The Runner-utilities now have more options for clearing</h3>
<p>Some of the options of <code>pyFoamClearCase.py</code> for clearing cases (for instance specifying additional files) have been ported to the <code>Runner</code>-utilities. Also is the <code>postProcessing</code>-directory removed by default</p>
<h2>Library</h2>
<h3>=SolutionDirectory= and <code>TimeDirectory</code> are more tolerant</h3>
<p>If there are field files and their zipped counterpart than instead of an error a warning <strong>can</strong> be given</p>
<h3>=ClusterJob= now handles template files</h3>
<p>A new method <code>templateFile</code> gets the name of a file which is constructed from a template of the same name plus the extension <code>.template</code></p>
<h3>Additional parameters in <code>ClusterJob</code></h3>
<p>The method <code>additionalParameters</code> can return a dictionary with additional parameters</p>
<h3>Custom data in directory easier accessible</h3>
<p>In the written data in the sub-dictionary <code>analyzed</code> there is now a subdictionary <code>Custom</code> with the values of the custom expressions with the prefix <code>CustomXX_</code> removed. This means that values that were available as</p>
<pre class="example"><code>data[&#39;Custom02_velCheck&#39;][&#39;min&#39;]
</code></pre>
<p>are now available as</p>
<pre class="example"><code>data[&#39;Custom&#39;][&#39;velCheck&#39;][&#39;min&#39;]
</code></pre>
<p>The advantage is that the number part which was dependent on the order the expressions were specified is now no longer necessary (this should make scripts more stable)</p>
<p>The old notation is still available but deprecated</p>
<h3>=SolverJob= now allows compression of output</h3>
<p>The parameter <code>solverLogCompress</code> compresses the log-file while writing it to disc. <strong>Attention:</strong> This may lead to corrupted log-files if the run crashes</p>
<h3>=PyFoamApplication=-class now allows quick access to data</h3>
<p>The dictionary returned by <code>getData()</code> now allows access to all the elements as attributes.</p>
<h2>New features/Utilities</h2>
<h3>Post-run hook that sends mail at the end of run</h3>
<p>The hook-module <code>MailToAddress</code> sends a mail at the end of a run. Prerequisite is an SMTP-Server that doesn't need authentication</p>
<h3>New utility <code>pyFoamCompressCases.py</code></h3>
<p>This utility goes through cases and compresses single files. The cases can be searched recursively to.</p>
<p>Purpose of this utility is to shrink cases where <code>writeCompression</code> was not turned on during the run</p>
<h3>Paraview-module to read additional data</h3>
<p>A new module <code>PyFoam.Paraview.Data</code> reads additional data usually written by OpenFOAM. These are converted to <code>vtkArray</code> using the following functions and can be used in <code>Programmable filters</code>:</p>
<dl>
<dt>setSampleData</dt>
<dd>reads the data from sampled sets
</dd>
<dt>setTimelineData</dt>
<dd>reads data from a timeline directory
</dd>
<dt>setPlotData</dt>
<dd>reads pickled plot data using <code>RedoPlot</code>
</dd>
</dl>
<h2>Enhancements</h2>
<h3>=pyFoamRedoPlot.py= can plot in XKCD-mode</h3>
<p>When used with the option <code>--implementation=xkcd</code> and version of <code>matplotlib</code> that supports it is installed then plots are done in the style of the webcomics <a href="http://xkcd.com">http://xkcd.com</a></p>
<h3>=pyFoamListCases.py= now displays disk usage in human readable form</h3>
<p>If the disk usage of the cases is calculated then it is displayed in human readable form (as KB, MB, GB or TB) for sizes larger than one Kilobyte</p>
<h3>=pyFoamClearCase.py= more flexible in selection of data to be removed</h3>
<p>Options to be more flexible in removing data are added:</p>
<dl>
<dt>keep</dt>
<dd>keep timesteps at a specified interval. For instance <code>--keep-interval=0.1</code> will keep times like <span class="math">1</span>, <span class="math">1.1</span> etc but remove <span class="math">1.05</span>
</dd>
<dt>keep</dt>
<dd>this will not remove any times in the <code>processor</code>-directories. Also are things like <code>keep-last</code> now honored for processor directories
</dd>
<dt>remove</dt>
<dd>Remove the directories with the analyzed data too. Old behavior was to remove them. Now they are kept by default
</dd>
</dl>
<h3>=pyFoamFromTemplate.py= automatically chooses template and default values</h3>
<p>If an output file <code>foo</code> is specified and no template then the utility looks for a file <code>foo.template</code> as a template.</p>
<p>If a file <code>foo.defaults</code> is present then this file is read and used as default parameter values. Other specifications override these defaults</p>
<h3>=pyFoamDumpRunDatabaseToCSV.py= can disable standard-fields</h3>
<p>Additional option <code>--disable-run-data</code></p>
<h3>=pyFoamDumpRunDatabaseToCSV.py= prints <code>pandas</code>-object</h3>
<p>With the <code>-pandas-print</code>-option a <code>DataFrame</code> is generated and printed</p>
<h3>Better debugging with <code>ipdb</code></h3>
<p>If the <code>ipdb</code>-package (basically <code>pdb</code> with <code>IPython</code>-additions) is installed then it is used. This gives additions like tab-completion</p>
<h3>Interactive shell after execution for utilities</h3>
<p>The option <code>--interactive-after-execution</code> drops the user to an interactive shell where the namespace can be inspected. If present <code>IPython</code> will be used, otherwise the regular shell is used</p>
<h3>Utilities that read quantitative data convert to <code>pandas</code>-data and/or <code>numpy</code></h3>
<p>This is mainly to be used on the interactive shell to do further analysis or write this data out. The utilities are:</p>
<dl>
<dt>pyDumpRunDatabaseToCSV</dt>
<dd>add an item <code>dump</code> with the whole data as a <code>DataFrame</code>
</dd>
<dt>pyFoamTimelinePlot</dt>
<dd>add element <code>series</code> with all the data as <code>Series</code> and <code>dataFrame</code> with the same data as a <code>DataFrame</code>
</dd>
<dt>pyFoamSamplePlot</dt>
<dd>Like <code>pyFoamTimelinePlot.py</code>
</dd>
<dt>pyFoamRedoPlot</dt>
<dd>Now can get series and the whole plot data as pandas-objects
</dd>
</dl>
<h3>Utilities that read quantitative data write Excel files</h3>
<p>The utilities <code>pyDumpRunDatabaseToCSV.py</code>, <code>pyFoamTimelinePlot.py</code>, <code>pyFoamSamplePlot.py</code> and <code>pyFoamRedoPlot.py</code> now have options to write Excel-files</p>
<h3>Specify additional settings for <code>GnuPlot</code> in <code>customRegexp</code></h3>
<p>If an item in <code>customRegexp</code> has an item <code>gnuplotCommands</code> then it is assumed that this is a list of strings which are executed before the first plotting. For instance</p>
<pre class="example"><code>          gnuplotCommands (
             &quot;set format y &#39;%.2e&#39;&quot;
           );
</code></pre>
<p>changes the number format on the y-axis</p>
<h3>More flexible data specification for <code>pyFoamSamplePlot.py</code></h3>
<p>Instead of determining the names of the fields and lines form the filenames it is now also possible to specify them through options.</p>
<p>The option <code>--is-distribution</code> is a shorthand that sets these options for distribution files</p>
<h3>=pyFoamSamplePlot.py= now allows specification of x-range</h3>
<p>The range of the x-axis of the plots can either be set by automatically scaling to the domains of all the data sets with <code>--scale-domain</code> or by specifying them with <code>--domain-minimum</code> or <code>--domain-maximum</code>.</p>
<p>These domains are set for <strong>all</strong> plots</p>
<h1>Version 0.6.1 - 2013-05-24</h1>
<h2>Major changes</h2>
<h2>Bugfixes</h2>
<h3>Restoring of <code>controlDict</code> after <code>write</code></h3>
<p>When activating an on-demand write the <code>constrolDict</code> was not restored because the output-line about the file being read was not recognized (due to a change in the output in recent OF-versions). Now a number of different formats is recognized</p>
<h3>Custom-plot type <code>slave</code> not working if no <code>master</code> defined</h3>
<p>That plot-type needs a <code>master</code>. Fixed to fail if none is defined</p>
<h3>=-list-only= did not correctly parse lists with a numeric prefix</h3>
<p>This did affect all utilities that use that option and also calls with <code>listOnly</code> to the library class</p>
<h2>Utilities</h2>
<h3>=pyFoamBuildHelper.py= now allow more than one action</h3>
<p>If multiple actions like <code>--update</code> and <code>--build</code> are specified they are executed in a sensible order (update before build etc)</p>
<h3>Utilities warn if OpenFOAM-version is unset</h3>
<p>If the environment variable that determines the OpenFOAM-version is unset a warning is issued by the utilities</p>
<h3>=pyFoamUpgradeDictionariesTo20.py= allows single files</h3>
<p>If single file is specified then the action to transform it has can be specified</p>
<h3>=pyFoamUpgradeDictionariesTo20.py= transforms reaction-schemes</h3>
<p>Now knows how to transform &quot;old&quot; reaction files (where the <code>reactions</code>-entry was a list) to the new format (where it is a dictionary). Only a limited number of reaction types is supported.</p>
<h3>=pyFoamUpgradeDictionariesTo20.py= transforms thermophysical data</h3>
<p>Now the old form of thermophysical data (lists) is transformed into the new dictionary-form</p>
<h3>=pyFoamCloneCase= now allows creating directory that symlinks to the original</h3>
<p>Now with the option <code>--symlink-mode</code> instead of copying the directories from the original new directories art created and populated with symlinks to the files in the original. The depth until which no symlinks to directories are created can be specified. This allows the clone to share the configuration files with the original</p>
<h3>=pyFoamClearCase.py= now removes <code>postProcessing</code> and allows removal of additional files</h3>
<p>The directory <code>postProcessing</code> is now automatically removed (can be switched off with <code>--keep-postprocessing</code>). Also with the <code>--additional</code>-option patterns with additional files to remove can be specified.</p>
<h3>Improvements to <code>pyFoamVersion.py</code></h3>
<ul>
<li>Now reports the location of the <code>python</code>-executable</li>
<li>Reports locations of used libraries</li>
</ul>
<h3>Additional files automatically cloned</h3>
<p>The files <code>Allrun</code>, <code>Allclean</code> and <code>0.org</code> are automatically added during cloning as these are often used by the standard-utilities</p>
<h3>=pyFoamDisplayBlockMesh.py= uses the same options for template format as <code>pyFoamFromTemplate.py</code></h3>
<p>This makes sure that templates are handled consistently and also allows different delimiters in the <code>blockMeshDict.template</code></p>
<h2>Library</h2>
<h3>Improvements in syntax of <code>ParsedParameterFile</code></h3>
<ul>
<li>Now the new relative scoping that was introduced in OF 2.2 is supported</li>
</ul>
<h3>=Utilities=-class now function to find files matching a pattern</h3>
<p>Added a function <code>find</code> that approxiamtes the <code>find</code>-command</p>
<h3>VCS ignores more files</h3>
<p>Some more patterns have been added that will be ignored in a VSC-controlled case. All of them concerning files that PyFoam creates during operation</p>
<h2>New features/Utilities</h2>
<h3>New Utility <code>pyFoamSymlinkToFile.py</code></h3>
<p>This utility replaces a symlink with a copy of the file/directories it points to. To be used after a <code>pyFoamCloneCase.py</code> in <code>--symlink-mode</code></p>
<h1>Version 0.6.0 - 2013-03-14</h1>
<h2>Major changes</h2>
<h3>Adaption to work with Python3</h3>
<p>Sources are adapted so that <code>PyFoam</code> works with Python3 too. This breaks support for Python 2.4 and earlier (possibly also Python 2.5)</p>
<p>Some of the Libraries in <code>PyFoam.ThirdParty</code> had to be adapted to work with Python3:</p>
<dl>
<dt>Gnuplot</dt>
<dd>The original version 1.8 is quite old. It was adapted with the help of the <code>six</code>-library (see below) to work with Python2 and Python3 (inspired by [<a href="https://github.com/oblalex/gnuplot.py-py3k/commits/master">https://github.com/oblalex/gnuplot.py-py3k/commits/master</a>] which is a pure port to Python3 without backwards compatibility)
</dd>
</dl>
<h3>New ThirdParty-Libraries</h3>
<dl>
<dt>six</dt>
<dd>Library that helps supporting Python 2 and Python 3 in the same source code. Currently version 1.2 from [<a href="https://bitbucket.org/gutworth/six">https://bitbucket.org/gutworth/six</a>] is used
</dd>
<dt>pyratemp</dt>
<dd>Templating library to support the new templating format. Version 0.2.0 from [<a href="http://www.simple-is-better.org/template/pyratemp.html">http://www.simple-is-better.org/template/pyratemp.html</a>] is used
</dd>
</dl>
<h3>Porting to <code>Windows</code></h3>
<p>Port to allow running PyFoam on Windows was done by Bruno Santos of blueCAPE (bruno.santos@bluecape.com.pt)</p>
<p>Patch was originally posted at <a href="http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=166">http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=166</a></p>
<p><strong>Note</strong>: many of PyFoam's features are not yet fully functional on Windows.</p>
<h3>Experimental port to <code>pypy</code></h3>
<p>Sources are executed in <code>pypy</code> but it seems there are problems with <code>numpy</code> and also with code like <code>for l in open(f).readlines()</code></p>
<h2>Third-Party</h2>
<h3>Upgraded <code>ply</code> to 3.4</h3>
<p>This brings virtually no changes. <code>README</code> with copyright information has been added</p>
<h2>Infrastructure</h2>
<h3>Parameters can't be modified in <code>CTestRun</code> after initialization</h3>
<p>This should help to avoid side-effects</p>
<h3>Treat timeouts in the <code>MetaServer</code> right</h3>
<p>Due to a previous workaround timeouts when collecting information about new machines was not treated correctly</p>
<h3>Add <code>execute</code>-method to <code>ClusterJob</code></h3>
<p>This allows the execution of a shell-script in the directory of the case</p>
<h3>Add possibility to run specific modules before or after the solver</h3>
<p>These modules are found in <code>PyFoam.Infrastructure.RunHooks</code>. Two concrete implementations:</p>
<dl>
<dt><code>PrintMessageHook</code></dt>
<dd>to print a text to the terminal
</dd>
<dt><code>SendToWebservice</code></dt>
<dd>encode an URL and send it to a webservice (example for <code>pushover.net</code> added)
</dd>
</dl>
<p>Hooks are automatically instantiated from the configuration data (examples are hardcoded))</p>
<h3>Parameters added to the info about the run</h3>
<p>The Runner-classes now have a parameter <code>parameters</code>. This data (usually it would be a dictionary) is added verbatim to the run info.</p>
<p>Most runner applications now have the possibility to add this info.</p>
<p>Purpose of this facility is to identify different runs in the database better.</p>
<h3>Parameter handling in <code>ClusterJob</code> extended</h3>
<p>Parameter values are now handed to the actual job. Also a dictionary with parameters can be handed to the constructor and will be used in the relevant callbacks</p>
<h3>Run data written alongside <code>PickledPlots</code></h3>
<p>During the run whenever the <code>PickledPlots</code>-file is written a file <code>pickledUnfinishedData</code> gets written. This has the current solver data and is similar to <code>pickledData</code>.</p>
<p>Also a file <code>pickledStartData</code> gets written that has the data that is available at the start of the run.</p>
<h3>=BasicRunner= collects error and warning texts</h3>
<p>The runner collects</p>
<ul>
<li>at every warning the next 20 lines of the output until a total of 500 lines is reached (this avoids filling disk and memory if the solver produces too many warnings)</li>
<li>All output from an error message until the end</li>
</ul>
<p>And stores them in the application data</p>
<h2>Library</h2>
<h3>=TemplateFile= now uses <code>pyratemp</code></h3>
<p>The class <code>TempalteFile</code> now uses an enhanced templating engine. The old implementation is in the class <code>TemplateFileOldFormat</code></p>
<h3>Clearer error message in Application-classes</h3>
<p>If used as classes (not as utilities) these classes print the class name instead of the calling utilities name</p>
<h3>Output is only colored if it goes to the terminal</h3>
<p>Error and warning messages don't decorate the output if it goes to files or other non-terminal streams</p>
<h3>=error=-method of application classes now raises an exception</h3>
<p>An exception is now raised by <code>self.error()</code>. This makes it easier to handle such errors if the application class is used. The exception is passed up until there is a &quot;real&quot; application</p>
<h3>=ParsedParameterFile= now knows how to handle binary files</h3>
<p>When the format of a file is <code>binary</code> lists with a length prefix are being read as binary blobs.</p>
<p>For reading the blobs a simple heuristics is used: a multiple of the length in bytes is read. If the next character is a <code>)</code> and the characters after that are a certain combination of characters (newlines and <code>;</code>) then it is assumed that the blob has ended. This may fail on certain occasions:</p>
<ul>
<li>if the combination of characters appears at these places</li>
<li>if the objects inside the binary data are of different sizes</li>
</ul>
<p>It would be hard to work around these restrictions without reprogramming the full functionality of OpenFOAM</p>
<h3>=LabledReSTTable= for more flexible table generation</h3>
<p>New class in the <code>RestructuredTextHelper</code> allows more flexible generation of tables. Items are added with <code>column</code> and <code>row</code> and if these don't exist in the first row/column the table is extended appropriately</p>
<h3>Plotting classes now allow setting of <code>xlabel</code></h3>
<p>This is implemented for <code>Gnuplot</code> and <code>Matplotlib</code>. Default for the label on the x-Axis is now &quot;Time [s]&quot;</p>
<h2>Utilities</h2>
<h3>=pyFoamFromTemplate.py= with new templating engine</h3>
<p>The utility can now use the pyratemp-templating engine which allows templates with loops, conditions and other fancy stuff</p>
<h3>=pyFoamSamplePlot.py= allows using the reference data as basis for comparison</h3>
<p>Instead of using the x-values from the original data the y-values of the reference data can be used for comparing (with the <code>--use-reference</code>-option)</p>
<p>Same for <code>pyFoamTimelimePlot.py</code></p>
<h3>Scaling and offsets are now used in plots of <code>pyFoamSamplePlot.py</code></h3>
<p>If scales not equal to <span class="math">1</span> and offsets not equal to <span class="math">0</span> are specified they are used in the <code>gnuplot</code>-output</p>
<h3>=pyFoamPrintData2DStatistics.py= prints relative average error</h3>
<p>With the <code>--relative-average-error</code>-option</p>
<h3>Enhancements to <code>pyFoamVersion.py</code></h3>
<ul>
<li>More tolerant if no library was found</li>
<li>Reports the location of the PyFoam-Library</li>
<li>Checks whether utility version is consistent the library found</li>
</ul>
<h3>=pyFoamRunner.py= allows hooks</h3>
<p>Hooks can be added at the start and the end of a run</p>
<h3>=pyFoamRedoPlots.py= supports range for plots</h3>
<p>Added <code>-end</code> and <code>-start</code>-option to select a range that should be plotted.</p>
<p>Currently not working with the Matplotlib-implementation (only gnuplot)</p>
<h3>=pyFoamDisplayBlockMesh.py= no supports templates</h3>
<p>If a file with values is specified then the utility assumes you're editing a template file and will evaluate it before displaying it</p>
<h3>=pyFoamCaseReport.py= is tolerant towards binary files</h3>
<p>New switch that makes the parser treat files that are declared <code>binary</code> in the header as if they were <code>ascii</code></p>
<h3>=pyFoamSamplePlot.py= and <code>pyFoamTimelinePlot.py</code> raise error if no plots are generated</h3>
<p>This makes it easier to catch faulty specifications (or empty timeline-files)</p>
<h3>=pyFoamSurfacePlot.py= can wait for a key</h3>
<p>An option <code>--wait</code> has been added that makes the utility wait before displaying the next picture</p>
<h3>=pyFoamEchoDictionary.py= is more flexible with binary files</h3>
<p>Switch allows forcing it to read a binary File as an ASCII</p>
<h3>All utilities now have a switch that starts the debugger even with syntax-errors</h3>
<p>Previously the option <code>--interactive-debug</code> only started the debugger if the error was <strong>no</strong> syntax error. This is still the default behavior, but can be overruled</p>
<h3>Utilities now can be killed with <code>USR1</code> and will give a traceback</h3>
<p>The option <code>--catch-USR1-signal</code> now installs a signal-handler that prints a traceback and finishes the run. If the interactive debugger is enabled then it goes to the debugger-shell.</p>
<p>Option <code>--keyboard-interrupt-trace</code> triggers the same behaviour for keyboard interrupts with <code>&lt;Ctrl&gt;-C</code></p>
<h3>Switch to switch on <strong>all</strong> debug options</h3>
<p>For the purpose of developing a switch <code>--i-am-a-developer</code> has been added.</p>
<h3>Plotting utilities now allow specification of x-Axis label</h3>
<p>With the option <code>xlabel</code> in the <code>customRegexp</code>-file the label on the x-axis of the plot can be changed. Setting <code>ylabel</code> and <code>y2label</code> (for the secondary axis) was already possible</p>
<h3>Metrics and compare for <code>pyFoamTimelinePlot.py</code> and <code>pyFoamSamplePlot.py</code> support time ranges</h3>
<p>Now the options <code>--min-time</code> and <code>--max-time</code> are supported by <code>--metrics</code> and <code>--compare</code></p>
<h3>=pyFoamDisplayBlockMesh.py= allows graphical selection of blocks and patches</h3>
<p>New addition by Marc Immer allows the graphical selection of blocks and patches and adds them to the <code>blockMeshDict</code></p>
<h3>=pyFoamCloneCase.py= and <code>pyFoamPackCase.py</code> accept additional parameters</h3>
<p>The file <code>LocalConfigPyFoam</code> is read by these utilities and if there is a parameter <code>addItem</code> in the section <code>Cloning</code> defined then these files are cloned/packed automatically (no user specification required)</p>
<h3>=pyFoamListCases.py= now calculates estimated end-times</h3>
<p>Additional option to print the estimated end times. These can be wrong if the case did not start from the <code>startTime</code> in the <code>controlDict</code>.</p>
<p>Also now allows printing the end and the start-time according to the <code>controlDict</code></p>
<h2>New features</h2>
<h3>Different &quot;phases&quot; for multi-region solvers</h3>
<p>Plots of type <code>phase</code> in <code>customRegexp</code> don't actually plot anything. The set a phase-name that is used for subsequent values (for instance to distinguish the different residuals)</p>
<h3>=pyFoamChangeBoundaryType.py= allows selection of region and time</h3>
<p>Options <code>--region</code> and <code>--time-directory</code> added that allow selecting different <code>boundary</code>-files</p>
<h3>New class for storing case data in a sqlite-database and associated utilities</h3>
<p>The class <code>RunDatabase</code> stores the data from runs. Utility <code>pyFoamAddCaseDataToDatabase.py</code> is one way to populate the database. <code>pyFoamDumpRunDatabaseToCSV.py</code> allows dumping that data to a file for further processing (in a spreadsheet for instance)</p>
<p>Database can also be populated using a special post-run hook</p>
<h2>Bugfixes</h2>
<h3>Only binary packages of 1.x were found</h3>
<p>Pattern had to start with 1 (now every digit is possible))</p>
<h3>Option group <em>Regular expressions</em> was listed twice</h3>
<p>No harm done. But fixed</p>
<h3>=--clear=-option for <code>pyFoamDecompose.py</code> not working</h3>
<p>Reason was that <code>rmtree</code> does not allow wildcards. Fixed</p>
<h3>=pyFoamDisplayBlockmesh.py= not working with variable substitution</h3>
<p>The <code>DictRedirect</code> would not convert to float. Fixed. Although it might happen again for other data types</p>
<h3>Option <code>--function-object-data</code> of <code>pyFoamClearCase.py</code> not working with directories</h3>
<p>The option was only implemented for the list-form of the <code>functions</code> entry in <code>controlDict</code></p>
<p>Now fixed to also work with the dictionary-form</p>
<h3>=nonuniform= of length 0 not correctly printed</h3>
<p>Seems like the length was interpreted as the name of the list. Fixed</p>
<h3>Building of pseudocases with <code>pyFoamRunner.py</code> broken</h3>
<p>Only worked if no region was specified (= not at all). Fixed</p>
<h3>=pyFoamRedoPlot.py= did not correctly honor <code>--end</code> and <code>--start</code></h3>
<p>Plots were over the whole data range. This is now fix (also the issue that <code>--end</code> alone did not work)</p>
<h3>=WriteParameterFile= does not preserve the order of additions</h3>
<p>Contents was &quot;only&quot; set as a dictionary which does not preserve the order in which entries are added. Replaced with a <code>DictProxy</code></p>
<h3>Wrong number of arguments when using <code>TimelinePlot</code> in <code>positions</code>-mode</h3>
<p>Problem that was introduced by changes in the <code>fields</code>-mode</p>
<h3>=ClusterJob= uses only <code>metis</code> for decomposition</h3>
<p>For OpenFOAM-versions 1.6 and higher the automatic decomposition used is now <code>scotch</code></p>
<h3>=pyFoamSamplePlot.py= and <code>pyFoamTimelinePlot.py</code> produced no pictures for regions</h3>
<p>As regions have their own subdirectories the <code>/</code> from the directory name was inserted into the filename and if the subdirectory did not exist <code>gnuplot</code> did not create the picture</p>
<h3>Barplots in <code>pyFoamTimelinePlot.py</code> not working if value is a vector</h3>
<p>The base class didn't correctly handle the <code>(</code> and <code>)</code>. Fixed</p>
<h3>Mysterious deadlocks while plotting long logfiles</h3>
<p>The problem was that during splitting the timeline data an exception was raised. This exception was caught by another part of PyFoam. This left a lock on the data structure locked and the next access to the structure was held indefinitely. Fixed</p>
<h3>Scanning linear expressions form the block coupled solver failed</h3>
<p>As there is a tuple of residuals the scanner did not analyze the output of the output of the block-coupled solver from <code>1.6-ext</code> correctly. This is now treated as a special case and each residual is plotted separately (distinguished by a <code>[x]</code> with <code>x</code> being the number of the component)</p>
<h3>=#include= not correctly working with macros in the included file</h3>
<p>Macros <code>$var</code> were not correctly expanded. Fixed</p>
<h3>Macros not correctly expanded to strings</h3>
<p>When being expanded to string form macros were not correctly expanded</p>
<h3>=pyFoamPackCase.py= in the working directory produces 'invisible' tar</h3>
<p>If the utility was used in the form</p>
<pre class="example"><code>pyFoamPackCase.py .
</code></pre>
<p>then an 'invisible' tar <code>..tgz</code> was produced. Fixed</p>
<h3>String at the end of a linear solver output makes parsing fail</h3>
<p>Reported in [<a href="http://www.cfd-online.com/Forums/openfoam-solving/112278-pyfoam-struggles-adopted-solver-post403990.html">http://www.cfd-online.com/Forums/openfoam-solving/112278-pyfoam-struggles-adopted-solver-post403990.html</a>] the string is assumed to be part of the iteration number. Fixed</p>
<h3>Paraview utilities not working with higher Paraview versions</h3>
<p>At least for PV 3.14 and 3.98 the way the version number is determined has changed and the PV-utilities failed. This has been fixed but is untested with old versions</p>
<h3>Camera settings not honored with <code>pyFoamPVSnapshot.py</code></h3>
<p>For the first rendered view Paraview automatically resets the camera. This has now been switched off (so the snapshot is rendered correctly)</p>
<h1>Version 0.5.7 - 2012-04-13</h1>
<h2>Parser improvements</h2>
<ul>
<li>Problem with nested comments</li>
<li>Parse code streams</li>
<li>Preserving of comments can be switched off</li>
<li>Ignore extra semicolons</li>
<li>Allows parsing lists of length 3 and 9 as lists and not as vectors and tensors</li>
<li>&quot;lookup redirection&quot; in OF-dictionaries now works</li>
</ul>
<h2>Utility improvements</h2>
<ul>
<li>pyFoamSamplePlot.py stops if the same directory is compared</li>
<li>pyFoamSamplePlot.py shows the location of the biggest difference</li>
<li>pyFoamSamplePlot.py allows only same ranges for comparisons</li>
<li>Generation of pickled-files can be switched of for runner-utilities</li>
<li>Frequency with which the pickled file is written is adapted (so that it doesn't use ALL the CPU-time)</li>
<li>pyFoamVersion.py improved (Version of Python is checked etc)</li>
<li>pyFoamRedoPlot.py: fixed parameter checking</li>
<li>pyFoamPotentialRunner.py: temporarily disable libs and functions</li>
<li>Only write last N loglines from Runner-utility</li>
<li>pyFoamClearCase.py: allow local specification of additional files that should be cleared</li>
<li>Runner utilities can report data about the run</li>
<li>pyFoamConvertToCSV.py now allows selection of columns</li>
<li>Utilities for quantative analysis now can return data</li>
<li>Utilities for quantative now correctly return data for multiple places</li>
<li>pyFoamSamplePlot.py now allows specification of valid variable pre and postfixes to allow correct parsing of variable names with a _</li>
<li>endTime can be specified by the runner-utilities</li>
<li>Utilities now allow piping (using pickled data)</li>
<li>pyFoamSamplePlot.py now allows the specification of a reference time</li>
<li>Nomenclature of pyFoamSamplePlot.py and pyFoamTimelinePlots.py now similar (both call it fields)</li>
<li>pyFoamDecompose.py now can use the -region-option ifthe OF-version is right</li>
<li>Forcing debug-mode fixed for older OF-versions</li>
<li>pyFoamDecomposer.py now accepts globalFaceZones in Python or OpenFOAM-syntax</li>
<li>Plot-utilities now don't interpret _ in names not as LaTeX</li>
</ul>
<h2>New Utilities</h2>
<ul>
<li>pyFoamEchoPickledApplicationData to output pickled data</li>
<li>pyFoamPrintData2DStatistics.py to output data from comparisons</li>
<li>pyFoamBuildHelper.py to build project and its prerequisites (work in progress)</li>
<li>pyFoamCreateModuleFile.py to create files for <a href="http://modules.sourceforge.net/">http://modules.sourceforge.net/</a> (by Martin Beaudoin)</li>
<li>pyFoamSTLUtility.py to join STL-files</li>
</ul>
<h2>Library improvements</h2>
<ul>
<li>stabler comparisons</li>
<li>Paraview-Utilities work with 1.x and 2.x</li>
<li>Runner classes return a dictionary with data</li>
<li>TimelineDirectory ignores dot-files</li>
<li>Application-objects can now be used like dictionaries to access data</li>
<li>New class TableData for simple data tables</li>
<li>New class Data2DStatistics for statistics about tables</li>
<li>new class CTestRun as basis for automated testing</li>
<li>FoamOptionParser now resets environment variables so that application-classes can call other application classes</li>
<li>Improvements to HgInterface</li>
<li>Additional VCS-subclasses for git, svn and svk (implementation only as needed)</li>
<li>SolutionDir now uses 0.org as initial directory if no valid initial directory is present (this affects clearing and cloning)</li>
<li>FoamFileGenerator now more flexible for long lists</li>
<li>ParsedBlockMeshDict now doesn't introduce prefixes for 'long' lists</li>
</ul>
<h2>Removed utilities</h2>
<ul>
<li>pyFoamAPoMaFoX.py</li>
<li>pyFoamPlotResiduals.py</li>
</ul>
<h2>Thirdparty</h2>
<ul>
<li>Got rid of Numeric-support in Gnuplot-library</li>
</ul>
<h2>Other</h2>
<ul>
<li>script to generate man-pages for the utilities</li>
<li>Paraview3-example probeDisplay.py now renamed to probeAndSetDisplay.py and reads sampledSets from controlDict and sampleDict</li>
<li>Older Versions The changes for older versions can be found on <a href="http://openfoamwiki.net/index.php/Contrib_PyFoam#History">the Wiki-page</a></li>
</ul>