Skip to main content

download_tidy_finance.get_available_huggingface_files()

List parquet files in a Hugging Face dataset.

Usage

Source

download_tidy_finance.get_available_huggingface_files(
    organization,
    dataset,
)

Queries the Hugging Face Datasets API and returns a data frame of files with a ‘.parquet’ suffix. The function follows pagination links returned in the response ‘Link’ header and returns the path and size for each file. Requires internet access and the dataset to be publicly accessible or accessible with appropriate authentication.

Parameters

organization: str

Hugging Face organization or user name.

dataset: str
Dataset name under the organization.

Returns

pl.DataFrame
A data frame with columns ‘path’ (str) and ‘size’ (int).

Examples

from tidyfinance import get_available_huggingface_files
get_available_huggingface_files('voigtstefan', 'sp500')