pelicun.file_io

Classes and methods that handle file input and output.

Functions

`_load_dlml_resource_paths`()	Load the map of method aliases to DLML dataset IDs.
`_resolve_default_dataset_id`(method_name)	Map a default method name to a DLML dataset ID.
`_unknown_method_msg`(method_name)	Describe an unrecognized default method name.
`load_data`(data_source[, ...])	Load data assuming it follows standard SimCenter tabular schema.
`load_from_file`(filepath[, log])	Load data from a file and stores it in a DataFrame.
`resolve_default_dataset_path`(method_name)	Resolve a default method name to its dataset folder.
`save_to_csv`(data, filepath[, units, ...])	Save data to a CSV file following the standard SimCenter schema.
`substitute_default_path`(data_paths[, log])	Substitute the default directory path.

pelicun.file_io.load_data(data_source: str | DataFrame, unit_conversion_factors: dict | None = None, orientation: int = 0, *, reindex: bool = True, return_units: bool = False, log: Logger | None = None) → tuple[DataFrame, Series] | DataFrame[source]

Load data assuming it follows standard SimCenter tabular schema.

The data is assumed to have a single header line and an index column. The second line may start with ‘Units’ in the index and provide the units for each column in the file.

Parameters:

data_source: string or DataFrame: If it is a string, the data_source is assumed to point to the location of the source file. If it is a DataFrame, the data_source is assumed to hold the raw data.
unit_conversion_factors: dict, optional: Dictionary containing key-value pairs of unit names and their corresponding factors. Conversion factors are defined as the number of times a base unit fits in the alternative unit. If no conversion factors are specified, then no unit conversions are made.
orientation: int, {0, 1}, default: 0: If 0, variables are organized along columns; otherwise they are along the rows. This is important when converting values to follow the prescribed units.
reindex: bool: If True, reindexes the table to ensure a 0-based, continuous index
return_units: bool: If True, returns the units as well as the data to allow for adjustments in unit conversion.
log: Logger: Logger object to be used. If no object is specified, no logging is performed.

Returns:

tuple

data: DataFrame: Parsed data.
units: Series: Labels from the data and corresponding units specified in the data. Units are only returned if return_units is set to True.

Raises:

TypeError: If data_source is neither a string nor a DataFrame, a TypeError is raised.

pelicun.file_io.load_from_file(filepath: str, log: Logger | None = None) → DataFrame[source]

Load data from a file and stores it in a DataFrame.

Currently, only CSV files are supported, but the function is easily extensible to support other file formats.

Parameters:

filepath: string: The location of the source file.
log: base.Logger, optional: Optional logger object.

Returns:

tuple

data: DataFrame: Data loaded from the file.
log: Logger: Logger object to be used. If no object is specified, no logging is performed.

Raises:

FileNotFoundError: If the filepath is invalid.
ValueError: If the file is not a CSV.

pelicun.file_io.resolve_default_dataset_path(method_name: str) → Path[source]

Resolve a default method name to its dataset folder.

The returned folder holds the model data files of the method’s dataset in the installed Damage and Loss Model Library (the dlml package). Use this function to check whether a method provides an optional file (e.g., ‘pelicun_config.py’) before reading it. To resolve files that are expected to exist, use substitute_default_path, which raises a descriptive error when a requested file is unavailable.

Parameters:

method_name: str: A method alias listed in the {base.pelicun_path}/resources/dlml_resource_paths.json file, or a DLML dataset ID (e.g., ‘seismic/building/component/FEMA P-58 2nd Edition’).

Returns:

Path: The path to the dataset’s folder in the installed DLML package.

Raises:

KeyError: If the method name is neither a method alias in dlml_resource_paths.json nor a valid DLML dataset ID.

Save data to a CSV file following the standard SimCenter schema.

The produced CSV files have a single header line and an index column. The second line may start with ‘Units’ in the index or the first column may be ‘Units’ to provide the units for the data in the file.

Parameters:

data: DataFrame: The data to save.
filepath: Path: The location of the destination file. If None, the data is not saved, but returned in the end.
units: Series, optional: Provides a Series with variables and corresponding units.
unit_conversion_factors: dict, optional: Dictionary containing key-value pairs of unit names and their corresponding factors. Conversion factors are defined as the number of times a base unit fits in the alternative unit.
orientation: int, {0, 1}, default 0: If 0, variables are organized along columns; otherwise, they are along the rows. This is important when converting values to follow the prescribed units.
use_simpleindex: bool, default True: If True, MultiIndex columns and indexes are converted to SimpleIndex before saving.
log: Logger, optional: Logger object to be used. If no object is specified, no logging is performed.

Returns:

DataFrame or None: If filepath is None, returns the DataFrame with potential unit conversions and reformatting applied. Otherwise, returns None after saving the data to a CSV file.

Raises:

ValueError: If units is not None but unit_conversion_factors is None.
ValueError: If writing to a file fails.
ValueError: If the provided file name does not have the .csv suffix.

pelicun.file_io.substitute_default_path(data_paths: list[str], log: Logger | None = None) → list[str][source]

Substitute the default directory path.

This function iterates over a list of data paths and replaces those with the ‘PelicunDefault/’ substring with the full paths to model files in the installed Damage and Loss Model Library (the dlml package). Default paths are expected to follow the PelicunDefault/method_name/model_type.extension structure. The method_name identifies the methodology through one of the aliases in the {base.pelicun_path}/resources/dlml_resource_paths.json file, or directly through a DLML dataset ID (e.g., seismic/building/component/FEMA P-58 2nd Edition). The model_type identifies the type of model requested. Currently, the following types are supported: ‘fragility’, ‘consequence_repair’, ‘loss_repair’. The extension is intended to identify ‘CSV’ files with model parameters and ‘JSON’ files with metadata. The model_type and extension strings are not limited to the supported values: any model_type.extension that identifies a file in the method’s folder resolves to that file. Every returned path points to an existing file; requesting a file that the method does not provide raises an error. To check whether a method provides an optional file, resolve the method’s dataset folder with resolve_default_dataset_path and look for the file there. Legacy placeholder filenames right after ‘PelicunDefault/’ (e.g., PelicunDefault/damage_DB_FEMA_P58_2nd.csv) are still recognized, trigger a deprecation warning, and are mapped to the corresponding method and model type.

Parameters:

data_paths: list of str: A list containing the paths to data files. These paths may include a placeholder directory ‘PelicunDefault/’ that needs to be substituted with the actual path to the data file in the installed Damage and Loss Model Library.
log: Logger: Logger object to be used. If no object is specified, no logging is performed.

Returns:

list of str: The updated paths, with every ‘PelicunDefault/’ entry replaced by the full path to the corresponding file in the installed DLML package.

Raises:

TypeError: If an element of data_paths is not a string.
KeyError: If the method name after ‘PelicunDefault/’ is neither a method alias in dlml_resource_paths.json nor a valid DLML dataset ID.
KeyError: If a bare filename right after ‘PelicunDefault/’ is not one of the legacy placeholder filenames preserved for backwards compatibility.
KeyError: If a default data path does not include a filename.
dlml.DatasetFileNotFoundError: If the requested file is not provided by the method’s dataset.

Notes

The function assumes that base.pelicun_path is properly initialized and points to the correct directory where resources are located.
If a path in the input list does not contain ‘PelicunDefault/’, the path is added to the output list unchanged.
Backslashes in ‘PelicunDefault’ paths are treated as path separators, so Windows-style and mixed-separator inputs (e.g., ‘PelicunDefault/FEMA P-58fragility.csv’) are accepted on every platform.

Examples

>>> data_paths = ['PelicunDefault/Hazus Hurricane/fragility.csv', 'data/file2.txt']
>>> substitute_default_path(data_paths)
['<site-packages>/dlml/data/hurricane/building/portfolio/'
  'Hazus v5.1 coupled/fragility.csv',
  'data/file2.txt']