utilities

Download and management utilities

Source code

General Methods

class IceAdvect.utilities.reify(wrapped)

Class decorator that puts the result of the method it decorates into the instance

IceAdvect.utilities.get_data_path(relpath: list | str | Path)

Get the absolute path within a package from a relative path

Parameters:

relpath: list, str or pathlib.Path

Relative path

IceAdvect.utilities.get_cache_path(relpath: list | str | Path | None = None, appname='advect')

Get the path to the user cache directory for an application

Parameters:

relpath: list, str, pathlib.Path or None

Relative path

appname: str, default ‘advect’

Application name

IceAdvect.utilities.import_dependency(name: str, extra: str = '', raise_exception: bool = False)

Import an optional dependency

Adapted from pandas.compat._optional::import_optional_dependency

Parameters:

name: str

Module name

extra: str, default “”

Additional text to include in the ImportError message

raise_exception: bool, default False

Raise an ImportError if the module is not found

Returns:

module: obj

Imported module

IceAdvect.utilities.dependency_available(name: str, minversion: str | None = None)

Checks whether a module is installed without importing it

Adapted from xarray.namedarray.utils.module_available

Parameters:

name: str

Module name

minversionstr, optional

Minimum version of the module

Returns:

availablebool

Whether the module is installed

IceAdvect.utilities.Path(filename: str | Path, *args, **kwargs)

Create a URL or pathlib.Path object

Parameters:

filename: str or pathlib.Path

File path or URL

class IceAdvect.utilities.URL(urlname: str | Path, *args, **kwargs)

Handles URLs similar to pathlib.Path objects

classmethod from_parts(parts: str | list | tuple)

Return a URL object from components

Parameters:

parts: str, list or tuple

URL components

joinpath(*pathsegments: list[str])

Append URL components to existing

Parameters:

pathsegments: list[str]

URL components to append

resolve()

Resolve the URL

is_file()

Boolean flag if path is a local file

is_dir()

Boolean flag if path is a local directory

exists()

Boolean flag if URL is valid

geturl()

String representation of the URL object

get(*args, **kwargs)

Get contents from URL

headers(*args, **kwargs)

Get headers from URL

load(*args, **kwargs)

Load JSON response from URL

ping(*args, **kwargs) → bool

Ping URL to check connection

query(*args, **kwargs)

List contents from URL

read(*args, **kwargs)

Open URL and read response

request(*args, **kwargs)

Make URL request

urlopen(*args, **kwargs)

Open URL and return response

property name

URL basename

property netloc

URL network location

property parent

URL parent path as a URL object

property parents

URL parents as a list of URL objects

property parts

URL parts as a tuple

property path

URL path

property scheme

URL scheme

property stem

URL stem

IceAdvect.utilities.get_hash(local: str | IOBase | Path, algorithm: str = 'md5', include_algorithm: bool = False)

Get the hash value from a local file or BytesIO object

Parameters:

local: obj, str or pathlib.Path

BytesIO object or path to file

algorithm: str, default ‘md5’

Hashing algorithm for checksum validation

include_algorithm: bool, default False

Include the algorithm name in the returned hash

IceAdvect.utilities.get_git_revision_hash(refname: str = 'HEAD', short: bool = False)

Get the git hash value for a particular reference

Parameters:

refname: str, default HEAD

Symbolic reference name

short: bool, default False

Return the shorted hash value

IceAdvect.utilities.get_git_status()

Get the status of a git repository as a boolean value

IceAdvect.utilities.url_split(s: str)

Recursively split a URL path into a list

Parameters:

s: str

URL string

IceAdvect.utilities.get_unix_time(time_string: str, format: str = '%Y-%m-%d %H:%M:%S')

Get the Unix timestamp value for a formatted date string

Parameters:

time_string: str

Formatted time string to parse

format: str, default ‘%Y-%m-%d %H:%M:%S’

Format for input time string

IceAdvect.utilities._create_default_ssl_context() → SSLContext

Creates the default SSL context

IceAdvect.utilities._create_ssl_context_no_verify() → SSLContext

Creates an SSL context for unverified connections

IceAdvect.utilities._set_ssl_context_options(context: SSLContext) → None

Sets the default options for the SSL context

IceAdvect.utilities.check_connection(HOST: str, context: ~ssl.SSLContext = <ssl.SSLContext object>, timeout: int = 20)

Check internet connection with http host

Parameters:

HOST: str

Remote http host

context: obj, default IceAdvect.utilities._default_ssl_context

SSL context for urllib opener object

timeout: int, default 20

Timeout in seconds for blocking operations

IceAdvect.utilities.http_list(HOST: str | list, timeout: int | None = None, context: ~ssl.SSLContext = <ssl.SSLContext object>, parser=<lxml.etree.HTMLParser object>, format: str = '%Y-%m-%d %H:%M', pattern: str = '', sort: bool = False, **kwargs)

List a directory on an Apache http Server

Parameters:

HOST: str or list

Remote http host path

timeout: int or NoneType, default None

Timeout in seconds for blocking operations

context: obj, default IceAdvect.utilities._default_ssl_context

SSL context for urllib opener object

parser: obj, default lxml.etree.HTMLParser()

HTML parser for lxml

format: str, default ‘%Y-%m-%d %H:%M’

Format for input time string

pattern: str, default ‘’

Regular expression pattern for reducing list

sort: bool, default False

Sort output list

Returns:

colnames: list

Column names in a directory

collastmod: list

Last modification times for items in the directory

IceAdvect.utilities.from_http(HOST: str | list, timeout: int | None = None, context: ~ssl.SSLContext = <ssl.SSLContext object>, local: str | ~pathlib.Path | None = None, hash: str = '', chunk: int = 16384, headers: dict = {}, verbose: bool = False, fid: object = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, label: str | None = None, mode: oct = 509, **kwargs)

Download a file from a http host

Parameters:

HOST: str or list

Remote http host path split as list

timeout: int or NoneType, default None

Timeout in seconds for blocking operations

context: obj, default IceAdvect.utilities._default_ssl_context

SSL context for urllib opener object

local: str, pathlib.Path or NoneType, default None

Path to local file

hash: str, default ‘’

MD5 hash of local file

chunk: int, default 16384

Chunk size for transfer encoding

headers: dict, default {}

Dictionary of headers to append from URL request

verbose: bool, default False

Print file transfer information

fid: object, default sys.stdout

Open file object for logging file transfers if verbose

label: str or None, default None

Label for logging file transfer information if verbose

mode: oct, default 0o775

Permissions mode of output local file

Returns:

remote_buffer: obj

BytesIO representation of file

IceAdvect.utilities.from_json(HOST: str | list, timeout: int | None = None, context: ~ssl.SSLContext = <ssl.SSLContext object>, headers: dict = {}) → dict

Load a JSON response from a http host

Parameters:

HOST: str or list

Remote http host path split as list

timeout: int or NoneType, default None

Timeout in seconds for blocking operations

context: obj, default IceAdvect.utilities._default_ssl_context

SSL context for urllib opener object

headers: dict, default {}

Dictionary of headers to append from URL request

Returns:

json_response: dict

JSON response