download.download_data_huggingface()
Download data from a Hugging Face dataset.
Usage
download.download_data_huggingface(
dataset=None, start_date=None, end_date=None, type=None, **kwargs
)Downloads data from a supported Hugging Face dataset. For ‘high_frequency_sp500’, parquet files are filtered by date range and row-bound. For ‘factor_library’, portfolio characteristics are selected via ’_filter_factor_library_grid’, the matching return data is downloaded, and the result is filtered to ‘start_date’ and ‘end_date’ when both are supplied. ‘factor_library_costs’ downloads the turnover, the trading cost, and the microcap share of each portfolio from January 1994, with the same filters and ‘ids’ as ‘factor_library’ and the grid joined in the same way. For ‘factor_library_grid’, the grid itself is returned via ‘download_factor_library_grid’. ‘factor_library_persistence’ describes how up to date each sorting variable is under each lag and rebalancing frequency, optionally for some ’sorting_variable’s only.
The factor library downloads can be pinned to a release: ‘revision’ selects the commit of the Hugging Face dataset ‘tidy-finance/factor-library’, which holds the returns and the costs, and ‘grid_revision’ the commit of ‘tidy-finance/factor-library-grid’, which holds the grid and the persistence. Both default to ‘main’.
For ‘dataset’ set to ‘factor_library’, the defaults below reflect one common portfolio construction choice but may not suit every research question; always verify that the selected combination matches the intended design. Supported filter columns and their defaults are:
- ‘sorting_variable’: optional. The firm characteristic used to sort stocks into portfolios, named like the Open Source Asset Pricing signals (e.g., ‘size’ for market equity, ‘bm’ for book-to-market). See the ‘factor_library_grid’ dataset for all values. No default is applied; when omitted, all sorting variables are returned (subject to the remaining defaults).
- ‘min_size_quantile’ (defaults to 0.2): fraction of the smallest stocks (by market cap) excluded from the portfolio universe; 0.2 drops the bottom 20%.
- ‘exclude_financials’ (defaults to False): whether to drop financial-sector stocks (SIC 6000-6799) from the universe.
- ‘exclude_utilities’ (defaults to False): whether to drop utility-sector stocks (SIC 4900-4999) from the universe.
- ‘exclude_negative_earnings’ (defaults to False): whether to drop firms with negative earnings before sorting.
- ‘sorting_variable_lag’ (defaults to ‘6m’): lag applied to the sorting variable before portfolio assignment: ‘1m’ (the timing of Open Source Asset Pricing), ‘3m’, ‘6m’, or ‘ff’ (Fama-French).
- ‘rebalancing’ (defaults to ‘monthly’): how frequently portfolios are reformed; ‘monthly’ or ‘annual’.
- ‘n_portfolios_main’ (defaults to 10): number of quantile groups (e.g., 10 for decile portfolios).
- ‘sorting_method’ (defaults to ‘univariate’): whether portfolios are formed on a single sort (‘univariate’) or on a double sort with size as the second variable (‘bivariate-dependent’ or ‘bivariate-independent’).
- ‘n_portfolios_secondary’ (defaults to None): number of size groups for the secondary sort. Required when ‘sorting_method’ is not ‘univariate’.
- ‘breakpoints_exchanges’ (defaults to ‘NYSE’): exchange(s) used to compute breakpoints; ‘NYSE’ uses only NYSE-listed stocks to define quantile cutoffs (the conventional Fama-French approach).
- ‘breakpoints_min_size_threshold’ (defaults to None): minimum size quantile of the stocks that set the main breakpoints (e.g., 0.2). None means no minimum-size screen is applied.
- ‘weighting_scheme’ (defaults to ‘VW’): return weighting within portfolios; ‘VW’ for value-weighted, ‘EW’ for equal-weighted, or ‘capped VW’ for value-weighted with capped weights.
Passing ‘None’ for any filter column removes that filter entirely, returning all values for that column (e.g., ‘min_size_quantile=None’ includes all size groups). Passing ‘[None]’ selects the missing level instead, e.g. ‘min_size_quantile=[None]’ for the portfolios without a size screen.
Parameters
dataset: str = None-
The dataset to download. Supported values are ‘high_frequency_sp500’, ‘factor_library’, ‘factor_library_costs’, ‘factor_library_grid’, and ‘factor_library_persistence’.
start_date: str or date = None-
Start date (inclusive) in ‘YYYY-MM-DD’ format. Used for ‘high_frequency_sp500’ to filter parquet files by date, and forwarded to ‘factor_library’ as a date-range lower bound. For ‘high_frequency_sp500’ defaults to the available sample’s start (‘2007-06-27’) when not supplied; for ‘factor_library’, None returns the full history.
end_date: str or date = None-
End date (inclusive) in ‘YYYY-MM-DD’ format. Used for ‘high_frequency_sp500’ to filter parquet files by date, and forwarded to ‘factor_library’ as a date-range upper bound. For ‘high_frequency_sp500’ defaults to the available sample’s end (‘2007-07-27’) when not supplied; for ‘factor_library’, None returns the full history.
type: str = None-
Deprecated. Use ‘dataset’ instead. If provided, emits a DeprecationWarning and strips any leading ‘hf_’ prefix.
**kwargs: dict-
For ‘dataset’ set to ‘factor_library’ or ‘factor_library_costs’: either named arguments used to filter the portfolio grid, or ‘ids=
’ to bypass the grid filter and download specific portfolios directly via ‘download_factor_library_ids’. Filter arguments take the form ‘column=value’, where ‘value’ may be a list or tuple to match multiple levels. Optionally pass ‘fill_all=True’ to leave unspecified columns unrestricted (default False, i.e., unspecified columns are fixed at the defaults listed above). Passing None for any parameter removes that filter entirely, returning all values for that column, and ‘[None]’ selects its missing level. Passing an unrecognised column name raises a ‘ValueError’. ‘ids’ cannot be combined with filter arguments. ‘revision’ and ‘grid_revision’ pin the releases of the returns (or costs) and of the grid. For ‘dataset’ set to ‘factor_library_grid’, ‘refresh=True’ downloads the grid again instead of returning the copy cached in the current session, and ‘grid_revision’ pins it. For ‘factor_library_persistence’, ‘sorting_variable’ and ‘grid_revision’ are accepted. ‘revision’ does not apply to the grid or the persistence and raises a ‘ValueError’. Ignored for ‘high_frequency_sp500’.
Returns
pl.DataFrame- A data frame with the downloaded data. For ‘high_frequency_sp500’, contains 5-second aggregated order-book snapshots filtered to the requested date range. For ‘factor_library’, contains the columns ‘id’, ‘date’, and ‘ret’ joined with the full grid metadata for the matched portfolio IDs.
Raises
ValueError- If ‘dataset’ is None, unsupported, or if invalid filter names are passed for the factor library.
Examples
from tidyfinance import download_data_huggingface
download_data_huggingface(
'high_frequency_sp500', '2007-07-26', '2007-07-27'
)
download_data_huggingface(
'factor_library',
sorting_variable='high52',
rebalancing='annual',
)
download_data_huggingface(
'factor_library', sorting_variable='assetgrowth', fill_all=True
)
download_data_huggingface(
'factor_library',
sorting_variable='size',
start_date='2000-01-01',
end_date='2020-12-31',
)
download_data_huggingface('factor_library', ids=[1, 2, 3])