7.3.1. Parflow Module

parflow module

Export Run() object

class parflow.ParflowBinaryReader(file: str, precompute_subgrid_info: bool = True, p: int = None, q: int = None, r: int = None, header: Mapping[str, Number] = None, read_sg_info: bool = False)

Bases: object

The ParflowBinaryReader, unsurprisingly, provides functionality for reading parflow binary files. It is designed to separate the header reading and metadata from subgrids from the reading of the underlying subgrid data in an efficient and flexible way. The ParflowBinaryReader only ever stores state about the header and/or subgrid headers. When reading data it is immediately returned to the user in the form of a numpy array. The ParflowBinaryReader implements a simple Context Manager so it is recommended to use with the standard idiom:

:: with ParflowBinaryReader(file) as pfb:

data = pfb.read_all_subgrids()

Parameters:

file – The pfb file to read
precompute_subgrid_info – Whether or not to precompute subgrid information. This defaults to True but can be turned off for reading multiple pfb files to reduce the amount of IO overhead when reading a sequence of pfb files. This computes the subgrid offset bytes, subgrid locations, subgrid indices, subgrid shapes, as well as subgrid coordinates and chunk sizes
p – The number of subgrids along the x dimension. This is an optional input, if it is not given we will try to precompute it.
q – The number of subgrids along the y dimension. This is an optional input, if it is not given we will try to precompute it.
r – The number of subgrids along the z dimension. This is an optional input, if it is not given we will try to precompute it.
header – A dictionary representing the header of the pfb file. This is an optional input, if it is not given we will read it from the pfb file directly.
read_sg_info – Whether or not to read subgrid information directly from the pfb file. Precalculating subgrid information does not always work correctly for some files, especially velocity files. This reads the subgrid offset bytes, subgrid indices and subgrid shapes, then computes the subgrid coordinates subgrid locations, and chunk sizes as normal

close()

compute_subgrid_info(): Computes the subgrid information

iloc_subgrid(idx: int) → ndarray

Read a subgrid at some scalar index.

Parameters:: idx – The index of the subgrid to read
Returns:: The data from the idx’th subgrid.

loc_subgrid(sg_p: int, sg_q: int, sg_r: int) → ndarray

Read a subgrid given it’s (sg_p, sg_q, sg_r) coordinate in the subgrid-grid.

Parameters:

sg_p – Index in the p subgrid to read.
sg_q – Index in the q subgrid to read.
sg_r – Index in the r subgrid to read.

Returns:

The data from the (sg_p, sg_q, sg_r)’th subgrid.

read_all_subgrids(mode: str = 'full', z_first: bool = True) → Iterable[ndarray] | ndarray

Read all of the subgrids in the file.

Parameters:

mode – Specifies how to arange the data from the subgrids before returning.
z_first – Whether the z dimension should be first. If true returned arrays have dimensions (‘z’, ‘y’, ‘x’) else (‘x’, ‘y’, ‘z’)

Returns:

A numpy array or iterable of numpy arrays, depending on how mode is set. If full the returned array will be of dimensions (nx, ny, nz). If flat the returned data will be a list of each of the subgrid arrays. If tiled the returned data will be a numpy array with dimensions (p, q, r) where each index of the array contains the subgrid data which also will be numpy array of floats with dimensions (sg_nx, sg_ny, sg_nz) where each of sg_nx, sg_ny, and sg_nz are the size of the subgrid array.

read_header(): Reads the header

read_subarray(start_x: int, start_y: int, start_z: int = 0, nx: int = 1, ny: int = 1, nz: int = None, z_first: bool = True) → ndarray

Read a subsection of the full pfb file. For an example of what happens here consider the following image:

:: +——-+——-+ | | | | x|xx | +——-+——-+ | x|xx | | x|xx | +——-+——-+

Where each of the borders of the big grid are the four subgrids (2,2) that we are trying to index data from. The data to be selected falls in each of these subgrids, as denoted by the ‘x’ marks.

Parameters:

start_x – The index to start at in the x dimension.
start_y – The index to start at in the y dimension.
start_z – The index to start at in the z dimension. This is optional, and if not provided is 0.
nx – The number of values to read in the x dimension. This is optional, and if not provided is 1.
ny – The number of values to read in the y dimension. This is optional, and if not provided is 1.
nz – The number of values to read in the z dimension. This is optional, and if not provided is None, which indicates to read all of the values.
z_first – Whether the z dimension should be first. If true returned arrays have dimensions (‘z’, ‘y’, ‘x’) else (‘x’, ‘y’, ‘z’)

Returns:

A nd array with shape (nx, ny, nz).

read_subgrid_header(skip_bytes: int = 64): Reads a subgrid header at the position skip_bytes

read_subgrid_info(): Read the header for each subgrid directly from the pfb file, rather than calculating it via precalculate_subgrid_info

class parflow.Run(name, basescript=None)

Bases: BaseRun

Main object that can be used to define a ParFlow simulation

Parameters:

name (str) – Name for the given run.
basescript (str) – Path to current file so the simulation execution and relative path will be assumed to have the script directory as their working directory. If not provided, the working directory will be the directory where the python executable was run from.

check_nans(working_directory, include_forcing=True)

Check the input files for NaNs.

Parameters:

working_directory – The working directory of the ParFlow run.
include_forcing (bool) – If set to True, forcing files are going to be checked.

Raises:

ValueError – If an input file contains a NaN.

clone(name)

Method to generate a clone of a run (for generating run ensembles, etc.)

This will return an identical run with the given name. See parflow/test/python/new_features/serial_runs/serial_runs.py for an example.

Parameters:: name (str) – Name of the new run.

property data_accessor: Return a DataAccessor to streamline access to numpy array of the various field this run is linked to.

dist(pfb_file, **kwargs)

Distribute a PFB file using the P/Q/R settings from the run or override them with the provided arguments.

We can also use the kwargs to set other properties such as:

NX, NY, NZ…

classmethod from_definition(file_path)

Method to generate a Run object from a file.

Currently accepts the following input file types:: yaml pfidb

Parameters:: file_path (str) – path to the file to read in
Returns:: A new Run object

get_name(): Returns name of run

run(working_directory=None, skip_validation=False, undist=False)

Method to run simulation

That method will automatically run the database write, validation and trigger the ParFlow execution while checking ParFlow output for any error.

If an error occurred a sys.exit(1) will be issue.

Parameters:

working_directory (str) – Path to write output files. If not provided, the default run working directory will be used. This also affect the relative FileNames.
skip_validation (bool) – Allow user to skip validation before running the simulation.
undist (bool) – undistribute PFB output files

set_name(new_name)

Setting new name for a run

Parameters:: new_name (str) – New name for run

undist(run_file=None, working_directory=None)

Undistribute a PFB file.

Currently this does not support the split file PFB file format.

Parameters:

run_file – The name of the PFIDB file. Defaults to the last run.
working_directory – The working directory of the ParFlow run. Defaults to the last run.

write(file_name=None, file_format='pfidb', working_directory=None)

Method to write database file to disk

Parameters:

file_name (str) – Name of the file to write. If not provided, the name of the run will be used.
file_format (str) – File extension which also represent the output format to use. (pfidb, yaml, json) The default is pfidb.

Returns:

The full path to the written file. (str): The full path of the run which is what should be

given to ParFlow executable.

Return type:

(str)

write_subsurface_table(file_name=None, working_directory=None): Write a table of subsurface properties assigned to each unit.

parflow.pf_test_file(file, correct_file, message, sig_digits=6)

Python version of the tcl pftestFile procedure.

Two file paths are given as the first two arguments. The function reads them into ndarrays and calls a comparison function (msig_diff) to check if the files differ by more than sig_digits significant digits in any coordinate. If they do, the function prints an error message and the coordinate in which the files have the greatest difference.

Parameters:

file (string) – path to test output file
correct_file (string) – path to reference output file
message (string) – message to print in case of test failure
sig_digits (int) – number of significant digits

Returns:

True if files differ in no more than sig_digits: significant digits in all coordinates. False otherwise.

Return type:

bool

Raises:

FileNotFoundError – If file or correct_file do not exist.
Exception – If the function fails to read either file into an ndarray.

parflow.pf_test_file_with_abs(file, correct_file, message, abs_value, sig_digits=6)

Python version of the tcl pftestFileWithAbs procedure.

Two file paths are given as the first two arguments. The function reads them into ndarrays and calls a comparison function (msig_diff) to check if the files differ by more than sig_digits significant digits in any coordinate. If they do, the function checks if the difference in that coordinate is greater than abs_value. If it is, it prints an error message and the coordinate in which the files have the greatest difference.

Parameters:

file (string) – path to test output file
correct_file (string) – path to reference output file
message (string) – message to print in case of test failure
abs_value (float) – threshold to determine if two files differ enough to fail the test
sig_digits (int) – number of significant digits

Returns:

True if files differ in no more than sig_digits: significant digits in all coordinates. False otherwise.

Return type:

bool

Raises:

FileNotFoundError – If file or correct_file do not exist.
Exception – If the function fails to read either file into an ndarray.

parflow.read_pfb(file: str, keys: dict = None, mode: str = 'full', z_first: bool = True, read_sg_info: bool = True)

Read a single pfb file, and return the data therein

Parameters:

file – The file to read.
keys –
A set of keys for indexing subarrays of the full pfb. Optional. This is mainly a trick for interfacing with xarray, but the format of the keys is:

:: {‘x’: {‘start’: start_x, ‘stop’: end_x},

’y’: {‘start’: start_y, ‘stop’: end_y}, ‘z’: {‘start’: start_z, ‘stop’: end_z}}
mode – The mode for the reader. See ParflowBinaryReader::read_all_subgrids for more information about what modes are available.
read_sg_info – Precalculating subgrid information does not always work correctly for some files, especially velocity files. If True, read subgrid info directly from the pfb file (slower, but more robust)

Returns:

An nd array containing the data from the pfb file.

parflow.read_pfb_sequence(file_seq: Iterable[str], keys=None, z_first: bool = True, z_is: str = 'z', read_sg_info: bool = False)

An efficient wrapper to read a sequence of pfb files. This approach is faster than looping over the read_pfb function because it caches the subgrid information from the first pfb file and then uses that to initialize all other readers.

Parameters:

file_seq – An iterable sequence of file names to be read.
keys –
A set of keys for indexing subarrays of the full pfb. Optional. This is mainly a trick for interfacing with xarray, but the format of the keys is:

:: {‘x’: {‘start’: start_x, ‘stop’: end_x},

’y’: {‘start’: start_y, ‘stop’: end_y}, ‘z’: {‘start’: start_z, ‘stop’: end_z}}
z_first – Whether the z dimension should be first. If true returned arrays have dimensions (‘z’, ‘y’, ‘x’) else (‘x’, ‘y’, ‘z’)
z_is – A descriptor of what the z axis represents. Can be one of ‘z’, ‘time’, ‘variable’. Default is ‘z’.
read_sg_info – Precalculating subgrid information does not always work correctly for some files, especially velocity files. If True, read subgrid info directly from the pfb file (slower, but more robust)

Returns:

An nd array containing the data from the files.

parflow.write_pfb(file, array, p=1, q=1, r=1, x=0.0, y=0.0, z=0.0, dx=1.0, dy=1.0, dz=1.0, z_first=True, dist=True, **kwargs)

Write a single pfb file. The data must be a 3D numpy array with float64 values. The number of subgrids in the saved file will be p * q * r. This is regardless of the number of subgrids in the PFB file loaded by the ParflowBinaryReader into the numpy array. Therefore, loading a file with ParflowBinaryReader and saving it with this method may restructure the file into a different number of subgrids if you change these values.

If dist is True then also write a file with the .dist extension added to the file_name. The .dist file will contain one line per subgrid with the offset of the subgrid in the .pfb file.

Parameters:

file – The name of the file to write the array to.
array – The array to write.
p – Number of subgrids in the x direction.
q – Number of subgrids in the y direction.
r – Number of subgrids in the z direction.
x – The length of the x-axis
y – The length of the y-axis
z – The length of the z-axis
dx – The spacing between cells in the x direction
dy – The spacing between cells in the y direction
dz – The spacing between cells in the z direction
z_first – Whether the z-axis should be first or last.
dist – Whether to write the distfile in addition to the pfb.
kwargs – Extra keyword arguments, primarily to eat unnecessary args by passing in a dictionary with **dict.