----------------------------------------------------------------------
This is the API documentation for the tidyfinance library.
----------------------------------------------------------------------


## Download data


download_data(domain: str = None, dataset: str = None, start_date: str = None, end_date: str = None, type: str = None, **kwargs) -> polars.dataframe.frame.DataFrame

Download and process data based on domain and dataset.

Downloads and processes data based on the specified domain (e.g.,
Fama-French factors, Global Q factors, or macro predictors), dataset,
and date range. The function checks whether the specified domain is
supported and then delegates to the appropriate function for
downloading and processing the data.

Parameters
----------
domain : str
    The domain of the dataset to download, given as one of the
    canonical names returned by 'list_supported_datasets()':
    'Fama-French', 'Global Q', 'Goyal-Welch', 'WRDS', 'Pseudo Data',
    'Index Constituents', 'FRED', 'Stock Prices',
    'Open Source Asset Pricing', 'Global Factor Data',
    'Pastor-Stambaugh', 'Stambaugh-Yuan', 'Tidy Finance'. The
    previous short names (e.g. 'famafrench', 'wrds', 'pseudo') are
    still accepted but deprecated and will be removed in a future
    release.
dataset : str, optional
    The specific dataset to download within the domain.
start_date : str, optional
    A character string or date in 'YYYY-MM-DD' format specifying
    the start date for the data. If not provided, the full dataset
    or a subset is returned, depending on the dataset type.
end_date : str, optional
    A character string or date in 'YYYY-MM-DD' format specifying
    the end date for the data. If not provided, the full dataset
    or a subset is returned, depending on the dataset type.
type : str, optional
    Deprecated. Use 'domain' and 'dataset' instead. If provided, a
    DeprecationWarning is emitted and the legacy type is
    translated to a ('domain', 'dataset') pair via
    'list_supported_datasets'.
**kwargs
    Additional arguments passed to specific download functions
    depending on 'domain'. For instance, if 'domain' is
    'Index Constituents', arguments are passed to
    '_download_data_constituents'. If 'domain' is
    'Global Factor Data', the 'dataset' argument and arguments
    such as 'region', 'factors', 'classification', 'frequency',
    and 'weighting' are passed to '_download_data_jkp'. If
    'domain' is 'Tidy Finance' and
    'dataset' is 'factor_library', arguments are either filter
    inputs (e.g., 'sorting_variable', 'rebalancing', 'fill_all') or
    an explicit 'ids' vector that bypasses the grid filter and
    downloads the specified portfolios directly via
    '_download_factor_library_ids'; see
    '_download_data_huggingface' for details.

Returns
-------
pl.DataFrame
    A data frame with processed data, including dates and the
    relevant financial metrics, filtered by the specified date
    range. For 'Stock Prices', dates are trading days in the
    exchange's local time zone (UTC when Yahoo Finance omits it),
    and both start_date and end_date are inclusive.

Examples
--------
```python
from tidyfinance import download_data
download_data(
    'Fama-French',
    'Fama/French 5 Factors (2x3) [Daily]',
    '2000-01-01',
    '2020-12-31',
)
download_data(
    'Goyal-Welch', 'monthly', '2000-01-01', '2020-12-31'
)
download_data('Index Constituents', index='DAX')
download_data('FRED', series=['GDP', 'CPIAUCNS'])
download_data('Stock Prices', symbols=['AAPL', 'MSFT'])
download_data(
    'Global Factor Data',
    region='usa',
    factors='mkt',
    start_date='2000-01-01',
    end_date='2020-12-31',
)
download_data(
    'Pastor-Stambaugh', '2020-01-01', '2020-12-31'
)
download_data(
    'Stambaugh-Yuan', 'monthly', '2015-01-01', '2016-12-31'
)
download_data(
    'Tidy Finance', 'risk_free', '2020-01-01', '2020-12-31'
)
download_data(
    'Tidy Finance',
    'high_frequency_sp500',
    '2007-07-26',
    '2007-07-27',
)
download_data(
    'Tidy Finance',
    'factor_library',
    sorting_variable='high52',
    rebalancing='annual',
)
download_data('Tidy Finance', 'factor_library', ids=[1, 2, 3])
download_data('Tidy Finance', 'factor_library_grid')
```


## Open Source


list_supported_indexes() -> polars.dataframe.frame.DataFrame

Return a DataFrame of supported financial indexes.

Each row corresponds to one index and pairs the index name with the
URL of an iShares CSV holdings file plus the number of header rows
to skip when parsing it. This table powers
'download_data_constituents'.

Returns
-------
polars.DataFrame
    A DataFrame with three columns:

    - 'index': name of the financial index (e.g. 'DAX', 'S&P 500').
    - 'url': URL of the CSV file with constituent holdings.
    - 'skip': number of preamble rows in the provider file. Kept for
      compatibility; the header row is now detected automatically.

Examples
--------
```python
from tidyfinance import list_supported_indexes
supported_indexes = list_supported_indexes()
print(supported_indexes)
```

get_available_famafrench_datasets()

Get the list of datasets available from the Fama/French data library.

Returns
-------
list
    A list of valid dataset names for use with download_data_factors_ff.


## Tidy Finance


tidyfinance.download_tidy_finance

Tidy Finance hosted data downloads for tidyfinance.


## WRDS


get_wrds_connection() -> object

Establish a connection to Wharton Research Data Services (WRDS).

Opens a SQLAlchemy connection to the WRDS PostgreSQL database. Credentials
are read from the 'WRDS_USER' and 'WRDS_PASSWORD' environment variables,
which can also be supplied through a '.env' file in the working directory
('load_dotenv' is called before the lookup). The connection uses the
'postgresql+psycopg2' driver, requires SSL, and targets
'wrds-pgdata.wharton.upenn.edu' on port 9737. Pool pre-ping is enabled so
stale connections are detected and recycled automatically.

Returns
-------
sqlalchemy.engine.Connection
    An open SQLAlchemy connection to the WRDS database. Pass it to any
    'download_data_wrds_*' helper or use it directly with
    'polars.read_database'. Close it via 'disconnect_connection' when
    done.

Raises
------
ValueError
    If 'WRDS_USER' or 'WRDS_PASSWORD' is not set in the environment or
    '.env' file. The 'load_wrds_credentials' helper performs this check
    before the connection is opened.
sqlalchemy.exc.OperationalError
    If the database host is unreachable or the credentials are rejected.

See Also
--------
set_wrds_credentials : Interactively prompts for credentials and writes
    them to a '.env' file in the project or home directory.
disconnect_connection : Closes the connection returned by this function.

Examples
--------
```python
import os
os.environ['WRDS_USER'] = 'your_username'
os.environ['WRDS_PASSWORD'] = 'your_password'
from tidyfinance import get_wrds_connection, disconnect_connection
con = get_wrds_connection()
# ... query WRDS via con ...
disconnect_connection(con)
```

disconnect_connection(connection: object) -> bool

Close an open WRDS database connection safely.

Attempts to close the supplied connection. Any exception raised by
the underlying driver is swallowed so the call can be used in
'finally' blocks without masking the original error.

Parameters
----------
connection : object
    The connection object to close. Typically the value returned by
    'get_wrds_connection'.

Returns
-------
bool
    'True' on successful disconnection, 'False' if the close call
    raised an exception.

Examples
--------
```python
from tidyfinance import get_wrds_connection, disconnect_connection
con = get_wrds_connection()
disconnect_connection(con)
```

load_wrds_credentials() -> tuple

Load WRDS credentials from environment variables or a '.env' file.

Reads 'WRDS_USER' and 'WRDS_PASSWORD' from the process environment.
If neither is set, also looks for a '.env' file in the working
directory via 'load_dotenv'.

Returns
-------
tuple of (str, str)
    '(wrds_user, wrds_password)' pair suitable for building a
    SQLAlchemy connection URL.

Raises
------
ValueError
    If either 'WRDS_USER' or 'WRDS_PASSWORD' is missing after the
    '.env' file has been loaded.

process_trace_data(trace_all: polars.dataframe.frame.DataFrame) -> polars.dataframe.frame.DataFrame

Clean Enhanced TRACE trade reports.

Applies the Dick-Nielsen cleaning protocol to the raw Enhanced
TRACE message stream, removing cancellations, corrections, and
reversals and producing one observation per executed corporate
bond trade.

Parameters
----------
trace_all : pl.DataFrame
    Raw Enhanced TRACE messages with the message-status columns
    used by the cleaning protocol ('trc_st', 'msg_seq_nb',
    'orig_msg_seq_nb', 'trd_rpt_dt', 'trd_rpt_tm',
    'trd_exctn_dt', 'trd_exctn_tm', 'cusip_id', 'entrd_vol_qt',
    'rptd_pr', 'rpt_side_cd', 'cntra_mp_id', 'asof_cd').

Returns
-------
pl.DataFrame
    Cleaned trade panel containing only executed trades with
    cancellations, corrections, and reversals already removed.

Notes
-----
Trades are cleaned under two regimes split by 2012-02-06, the date
the Enhanced TRACE message-status format changed. Trades reported
on or after this cutoff are cleaned with the post-2012 logic
(status flags 'T', 'R', 'X', 'C', 'Y'); earlier trades use the
pre-2012 logic (status flags 'T', 'C', 'W' plus 'asof_cd'
reversals). The cutoff date follows Dick-Nielsen (2014).

References
----------
Dick-Nielsen, J. (2009). Liquidity biases in TRACE. Journal of
Fixed Income, 19(2), 43-55.
https://doi.org/10.3905/jfi.2009.19.2.043

Dick-Nielsen, J. (2014). How to clean enhanced TRACE data. Working
Paper. https://ssrn.com/abstract=2337908

set_wrds_credentials() -> None

Set WRDS credentials in a '.env' file.

Prompts interactively for the WRDS username and password and writes
them to a '.env' file as 'WRDS_USER' and 'WRDS_PASSWORD'. The
location is chosen at the prompt: 'project' writes to the current
working directory, 'home' writes to the user's home directory. If
a '.env' file already contains WRDS credentials, the user is asked
before overwriting. After saving, the user is offered to append
'.env' to a sibling '.gitignore' (recommended).

The resulting '.env' file is the credentials source consumed by
'get_wrds_connection' via 'load_wrds_credentials'.

Returns
-------
None
    Called for its side effects: writing '.env' and, optionally,
    updating '.gitignore'.

See Also
--------
get_wrds_connection : Opens a WRDS connection using the credentials
    stored by this function.

Examples
--------
```python
from tidyfinance import set_wrds_credentials
set_wrds_credentials()
```


## Portfolio analysis


data_options(id: str = 'permno', date: str = 'date', exchange: str = 'exchange', mktcap_lag: str = 'mktcap_lag', ret_excess: str = 'ret_excess', portfolio: str = 'portfolio', siccd: str = 'siccd', price: str = 'prc_adj', listing_age: str = 'listing_age', be: str = 'be', earnings: str = 'ib', **kwargs) -> dict

Create data options for tidyfinance functions.

Creates a dict of data options used by tidyfinance-related
functions. These options map the specific data variables to the
Tidy Finance naming conventions, allowing functions to flexibly
work with different datasets by specifying the relevant column
names. Additional options can be passed through '**kwargs'.

Parameters
----------
id : str, default "permno"
    Entity identifier column.
date : str, default "date"
    Date column.
exchange : str, default "exchange"
    Exchange column.
mktcap_lag : str, default "mktcap_lag"
    Market capitalization lag column.
ret_excess : str, default "ret_excess"
    Excess return column.
portfolio : str, default "portfolio"
    Portfolio assignment column.
siccd : str, default "siccd"
    SIC code column.
price : str, default "prc_adj"
    Adjusted price column.
listing_age : str, default "listing_age"
    Listing age column.
be : str, default "be"
    Book equity column.
earnings : str, default "ib"
    Earnings column (Compustat income before extraordinary items).
**kwargs
    Any additional column mappings stored verbatim in the dict.

Returns
-------
dict
    Mapping with at least the 11 standard column-name keys plus any
    extras provided via '**kwargs'.

Examples
--------
```python
from tidyfinance import data_options
data_options(id='permno', date='date', exchange='exchange')
```

assign_portfolio(data: polars.dataframe.frame.DataFrame, sorting_variable: str, breakpoint_options: dict = None, breakpoint_function=None, data_options: dict = None) -> polars.series.series.Series

Assign data points to portfolios based on a sorting variable.

Users may pass a custom function to compute breakpoints. The
function must take 'data' and 'sorting_variable' as the first two
arguments, then 'breakpoint_options' and 'data_options'. The
function must return an ascending sequence of breakpoints.
Defaults to compute_breakpoints.

Parameters
----------
data : pl.DataFrame
    Dataset for portfolio assignment.
sorting_variable : str
    Column in 'data' used for sorting and portfolio assignment.
breakpoint_options : dict, optional
    Named arguments passed to 'breakpoint_function'. Typically
    produced by breakpoint_options.
breakpoint_function : callable, optional
    Function to compute breakpoints. Must return an ascending
    sequence. Defaults to compute_breakpoints.
data_options : dict, optional
    Column-name mapping (see data_options). Passed through to
    'breakpoint_function'.

Returns
-------
pl.Series
    Portfolio assignments as a Float64 series aligned with the
    input rows. Each entry is the 1-indexed portfolio number;
    values outside the breakpoint range fall into the boundary
    portfolios. Missing inputs are returned as null.

Examples
--------
```python
import numpy as np
import polars as pl
from tidyfinance import assign_portfolio, breakpoint_options
rng = np.random.default_rng(42)
data = pl.DataFrame({
    'exchange': rng.choice(['NYSE', 'NASDAQ'], 100),
    'market_cap': rng.uniform(1, 100, 100),
})
# Quintile portfolios on market_cap with NYSE breakpoints
assign_portfolio(
    data,
    sorting_variable='market_cap',
    breakpoint_options=breakpoint_options(
        n_portfolios=5,
        breakpoints_exchanges='NYSE',
    ),
)
```

breakpoint_options(n_portfolios: int = None, percentiles: list = None, breakpoints_exchanges=None, smooth_bunching: bool = False, breakpoints_min_size_threshold: float = None, **kwargs) -> dict

Create breakpoint options for portfolio sorting.

Generates a structured dict of options for defining breakpoints in
portfolio sorting. It includes parameters for the number of
portfolios, percentile thresholds, exchange-specific breakpoints,
and smooth bunching, along with additional optional parameters.

Parameters
----------
n_portfolios : int, optional
    Number of portfolios to create. Must be a positive integer. If
    not provided, defaults to None.
percentiles : list of float, optional
    Percentile thresholds for defining breakpoints. Each value must
    be between 0 and 1. If not provided, defaults to None.
breakpoints_exchanges : str or list of str, optional
    Non-empty exchange (or list of exchanges) from which to compute
    the breakpoints. If not provided, defaults to None.
smooth_bunching : bool, default False
    Indicates whether smooth bunching should be applied.
breakpoints_min_size_threshold : float, optional
    When set to a value between 0 and 1, stocks with market
    capitalization below this quantile are excluded from breakpoint
    computation. The quantile is computed among
    'breakpoints_exchanges' stocks if specified, otherwise among
    all stocks. Requires a market capitalization column in the data
    (see 'data_options'). Defaults to None (no size filtering).
**kwargs
    Additional optional arguments. These will be captured in the
    resulting structure as part of the dict.

Returns
-------
dict
    Dictionary containing the provided breakpoint options,
    including any additional arguments passed via '**kwargs'.

Examples
--------
```python
from tidyfinance import breakpoint_options
# Quintile portfolios with NYSE breakpoints
breakpoint_options(
    n_portfolios=5,
    breakpoints_exchanges='NYSE',
)
# Custom percentile thresholds (mutually exclusive with n_portfolios)
breakpoint_options(
    percentiles=[0.3, 0.7],
    breakpoints_exchanges='NYSE',
)
# Exclude the smallest 20% of NYSE stocks from breakpoint computation
breakpoint_options(
    n_portfolios=10,
    breakpoints_exchanges='NYSE',
    breakpoints_min_size_threshold=0.2,
)
```

compute_breakpoints(data: polars.dataframe.frame.DataFrame, sorting_variable: str, breakpoint_options: dict, data_options: dict = None) -> numpy.ndarray

Compute breakpoints based on a sorting variable.

Computes breakpoints based on a specified sorting variable. It can
optionally filter the data by exchanges or lagged size quantiles
before computing the breakpoints. The function requires either the
number of portfolios to be created or specific percentiles for the
breakpoints, but not both. The function also optionally handles
cases where the sorting variable clusters on the edges, by
assigning all extreme values to the edges and attempting to compute
equally populated breakpoints with the remaining values.

Parameters
----------
data : pl.DataFrame
    DataFrame with the dataset for breakpoint computation.
sorting_variable : str
    Column name in 'data' to be used for determining breakpoints.
breakpoint_options : dict
    Named dict of 'breakpoint_options' for the breakpoints. The
    accepted entries include:

    - 'n_portfolios' (int, optional): Number of equally sized
      portfolios to create. Mutually exclusive with 'percentiles'.
    - 'percentiles' (list of float, optional): Percentiles defining
      the breakpoints of the portfolios. Mutually exclusive with
      'n_portfolios'.
    - 'breakpoints_exchanges' (str or list of str, optional):
      Exchange names to filter the data before computing
      breakpoints. Exchanges must be stored in the column given by
      'data_options' (defaults to 'exchange'). If None, no
      filtering is applied.
    - 'smooth_bunching' (bool, optional): Whether to attempt
      smoothing non-extreme portfolios if the sorting variable
      bunches on the extremes (True) or not (False, the default).
      In some cases, smoothing will not result in equal-sized
      portfolios off the edges due to multiple clusters. If
      sufficiently large bunching is detected, 'percentiles' is
      ignored and equally-spaced portfolios are returned for these
      cases with a warning.
    - 'breakpoints_min_size_threshold' (float, optional): Value
      between 0 and 1 (exclusive). When set, stocks with market
      capitalization below this quantile are excluded from
      breakpoint computation. The quantile is computed among
      'breakpoints_exchanges' stocks if specified, otherwise among
      all stocks. Requires a market capitalization column in the
      data (column name determined by 'data_options').

data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'exchange' key is
    used to specify the exchange column, and 'mktcap_lag' is used
    to specify the market capitalization column. Uses the
    'data_options' default when None: 'exchange' -> 'exchange' and
    'mktcap_lag' -> 'mktcap_lag'.

Returns
-------
np.ndarray
    Sorted array of breakpoints of the desired length.

Notes
-----
This function raises a ValueError if both 'n_portfolios' and
'percentiles' are provided or missing simultaneously.

Examples
--------
```python
import numpy as np
import polars as pl
from tidyfinance import compute_breakpoints, breakpoint_options
rng = np.random.default_rng(42)
data = pl.DataFrame({
    'id': range(1, 101),
    'exchange': rng.choice(['NYSE', 'NASDAQ'], 100),
    'market_cap': range(1, 101),
})
compute_breakpoints(
    data, 'market_cap', breakpoint_options(n_portfolios=5)
)
compute_breakpoints(
    data,
    'market_cap',
    breakpoint_options(
        percentiles=[0.2, 0.4, 0.6, 0.8],
        breakpoints_exchanges=['NYSE'],
    ),
)
```

filter_options(exclude_financials: bool = False, exclude_utilities: bool = False, min_stock_price: float = None, min_size_quantile: float = None, min_listing_age: float = None, exclude_negative_book_equity: bool = False, exclude_negative_earnings: bool = False, **kwargs) -> dict

Create filter options for sample construction.

Creates a dict of filter options used for sample construction in
tidyfinance-related functions. These options control which
observations are retained before portfolio sorting.

Parameters
----------
exclude_financials : bool, default False
    Whether to exclude financial firms (SIC codes 6000 to 6799).
exclude_utilities : bool, default False
    Whether to exclude utility firms (SIC codes 4900 to 4999).
min_stock_price : float, optional
    Minimum stock price required to include an observation. Must
    be strictly positive when provided. None (the default) applies
    no price filter.
min_size_quantile : float, optional
    Minimum cross-sectional size quantile (based on lagged market
    cap) required to include an observation. Must be strictly
    between 0 and 1 when provided. The cutoff is computed from
    NYSE stocks only; this requires an 'exchange' column in the
    data (as mapped via 'data_options'). None (the default)
    applies no size quantile filter.
min_listing_age : float, optional
    Minimum number of months a stock must have been listed in
    CRSP. Must be non-negative when provided. None (the default)
    applies no listing age filter.
exclude_negative_book_equity : bool, default False
    Whether to exclude observations with non-positive book equity.
exclude_negative_earnings : bool, default False
    Whether to exclude observations with non-positive earnings.
**kwargs
    Additional optional arguments, stored verbatim in the dict.

Returns
-------
dict
    Dict containing the specified filter options.

Examples
--------
```python
from tidyfinance import filter_options
filter_options(
    exclude_financials=True,
    exclude_utilities=True,
    min_stock_price=1,
    min_listing_age=12,
)
```

portfolio_sort_options(filter_options: dict = None, breakpoint_options_main: dict = None, breakpoint_options_secondary: dict = None, **kwargs) -> dict

Create portfolio sort options.

Creates a dict of options that bundles sample construction filters
and breakpoint specifications for use with
'implement_portfolio_sort'.

Parameters
----------
filter_options : dict, optional
    Dict produced by 'filter_options', or None (the default, which
    applies no filters). The accepted entries include:

    - 'exclude_financials' (bool): Whether to exclude financial
      firms (SIC codes 6000 to 6799). Defaults to False.
    - 'exclude_utilities' (bool): Whether to exclude utility firms
      (SIC codes 4900 to 4999). Defaults to False.
    - 'min_stock_price' (float, optional): Minimum stock price
      required to include an observation. None (the default)
      applies no price filter.
    - 'min_size_quantile' (float, optional): Minimum cross-sectional
      size quantile (based on lagged market cap) required to
      include an observation. None (the default) applies no size
      quantile filter.
    - 'min_listing_age' (float, optional): Minimum number of months
      a stock must have been listed in CRSP. None (the default)
      applies no listing age filter.
    - 'exclude_negative_book_equity' (bool): Whether to exclude
      observations with non-positive book equity. Defaults to False.
    - 'exclude_negative_earnings' (bool): Whether to exclude
      observations with non-positive earnings. Defaults to False.

breakpoint_options_main : dict, optional
    Dict produced by 'breakpoint_options', specifying breakpoints
    for the primary sorting variable, or None (the default) when
    no primary breakpoints are required. The accepted entries
    include:

    - 'n_portfolios' (int, optional): Number of equally sized
      portfolios. Mutually exclusive with 'percentiles'.
    - 'percentiles' (list of float, optional): Percentiles for
      defining the breakpoints. Mutually exclusive with
      'n_portfolios'.
    - 'breakpoints_exchanges' (str or list of str, optional):
      Exchange names to filter the data before computing
      breakpoints. If None, no filtering is applied.
    - 'smooth_bunching' (bool, optional): Whether to attempt
      smoothing non-extreme portfolios if the sorting variable
      bunches on the extremes.
    - 'breakpoints_min_size_threshold' (float, optional): Value
      between 0 and 1 (exclusive) below which stocks are excluded
      from breakpoint computation.

breakpoint_options_secondary : dict, optional
    Dict produced by 'breakpoint_options', specifying breakpoints
    for the secondary sorting variable, or None (the default) for
    univariate sorts. The accepted entries are the same as for
    'breakpoint_options_main'.
**kwargs
    Additional optional arguments, stored verbatim in the dict.

Returns
-------
dict
    Dict containing the specified options.

Examples
--------
```python
from tidyfinance import (
    portfolio_sort_options,
    filter_options,
    breakpoint_options,
)
portfolio_sort_options(
    filter_options=filter_options(exclude_financials=True),
    breakpoint_options_main=breakpoint_options(n_portfolios=10),
)
```

compute_portfolio_returns(data: polars.dataframe.frame.DataFrame, sorting_variables, sorting_method: str, rebalancing_month: int = None, breakpoint_options_main: dict = None, breakpoint_options_secondary: dict = None, breakpoint_function_main=None, breakpoint_function_secondary=None, min_portfolio_size: int = 1, cap_weight: float = 0.8, data_options: dict = None, quiet: bool = False) -> polars.dataframe.frame.DataFrame

Compute portfolio returns.

Computes individual portfolio returns based on specified sorting
variables and sorting methods. The portfolios can be rebalanced
every period or on an annual frequency by specifying a rebalancing
month, which is only applicable at a monthly return frequency. The
function supports univariate and bivariate sorts, with the latter
supporting dependent and independent sorting methods.

The function checks for consistency in the provided arguments. For
univariate sorts, a single sorting variable and a corresponding
number of portfolios must be provided. For bivariate sorts, two
sorting variables and two corresponding numbers of portfolios (or
percentiles) are required. The sorting method determines how
portfolios are assigned and how returns are computed. The function
handles missing and extreme values appropriately based on the
specified sorting method and rebalancing frequency.

Parameters
----------
data : pl.DataFrame
    Stock-level panel. Must contain the id, date, and excess
    return columns (configurable via 'data_options'), the sorting
    variable(s), and optionally a market-cap lag column for
    value-weighted returns.
sorting_variables : str or list of str
    Column name(s) in 'data' to use for sorting and portfolio
    assignment. For univariate sorts, provide a single variable.
    For bivariate sorts, provide two variables, where the first
    string refers to the main variable and the second string
    refers to the secondary ('control') variable.
sorting_method : {'univariate', 'bivariate-dependent',
                  'bivariate-independent'}
    Sorting method to use. For bivariate sorts, the portfolio
    returns are averaged over the controlling sorting variable
    (i.e., the second sorting variable), and only portfolio
    returns for the main sorting variable are returned.
rebalancing_month : int, optional
    Integer between 1 and 12 specifying the month in which to form
    portfolios that are held constant for one year. For example,
    setting it to 7 creates portfolios in July that are held
    constant until June of the following year. The default None
    corresponds to periodic rebalancing.
breakpoint_options_main : dict
    Named dict of 'breakpoint_options' passed to
    'breakpoint_function_main' for the main sorting variable.
    Required.
breakpoint_options_secondary : dict, optional
    Named dict of 'breakpoint_options' passed to
    'breakpoint_function_secondary'. Required for bivariate sorts.
breakpoint_function_main : callable, optional
    Function to compute breakpoints for the main sorting variable.
    Defaults to 'compute_breakpoints'.
breakpoint_function_secondary : callable, optional
    Function to compute breakpoints for the secondary sorting
    variable. Defaults to 'compute_breakpoints'.
min_portfolio_size : int, default 1
    Minimum number of firms required in the reported portfolio
    cross-section on a given date. For univariate sorts that is
    firms per portfolio-date; for bivariate sorts that is firms
    per main-portfolio-date summed across the secondary buckets.
    Cross-sections below the threshold have their returns set to
    null. A typical value is 5 (the Fama-French convention). Set
    to 0 to deactivate the check entirely.
cap_weight : float, default 0.8
    Quantile of the cross-sectional 'mktcap_lag' distribution at
    which market capitalizations are capped per date when computing
    the capped value-weighted excess return ('ret_excess_vw_capped').
    Must be in [0, 1].
data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'id', 'date',
    'ret_excess', and 'mktcap_lag' elements are used. Uses
    'data_options' defaults when None.
quiet : bool, default False
    If True, suppress informational warnings about missing
    observations in the output panel.

Returns
-------
pl.DataFrame
    Data frame with computed portfolio returns as a complete panel
    (all portfolio-date combinations), containing:

    - 'portfolio': Portfolio identifier.
    - date column (as in 'data_options'): Date of the portfolio
      return.
    - 'ret_excess_vw': Value-weighted excess return (only if 'data'
      contains the market-cap lag column). Null if insufficient
      observations.
    - 'ret_excess_ew': Equal-weighted excess return. Null if
      insufficient observations.
    - 'ret_excess_vw_capped': Capped value-weighted excess return
      (only if 'data' contains the market-cap lag column). Weights
      are computed using market capitalization capped at the
      'cap_weight' percentile per date. Null if insufficient
      observations.

Notes
-----
Ensure that 'data' contains all required columns: the specified
sorting variables and excess returns (see options and defaults set
in 'data_options'). A ValueError is raised if any required column
is missing.

Examples
--------
```python
import datetime as dt
import numpy as np
import polars as pl
from tidyfinance import (
    compute_portfolio_returns,
    breakpoint_options,
)
rng = np.random.default_rng(42)
dates = pl.date_range(
    dt.date(2020, 1, 1), dt.date(2028, 4, 1), '1mo', eager=True
)
data = pl.DataFrame({
    'permno': range(1, 501),
    'date': dates.to_numpy().repeat(5),
    'mktcap_lag': rng.uniform(100, 1000, 500),
    'ret_excess': rng.standard_normal(500),
    'size': rng.uniform(50, 150, 500),
})
# Univariate sorting with periodic rebalancing
compute_portfolio_returns(
    data, 'size', 'univariate',
    breakpoint_options_main=breakpoint_options(n_portfolios=5),
)
# Bivariate dependent sorting with annual rebalancing
compute_portfolio_returns(
    data, ['size', 'mktcap_lag'], 'bivariate-dependent', 7,
    breakpoint_options_main=breakpoint_options(n_portfolios=5),
    breakpoint_options_secondary=breakpoint_options(n_portfolios=3),
)
```

compute_long_short_returns(data: polars.dataframe.frame.DataFrame, direction: str = 'top_minus_bottom', data_options: dict = None) -> polars.dataframe.frame.DataFrame

Compute long-short returns.

Calculates long-short returns based on the returns of portfolios.
The long-short return is computed as the difference between the
returns of the 'top' and 'bottom' portfolios. The direction of the
calculation can be adjusted based on whether the return from the
'bottom' portfolio is subtracted from or added to the return from
the 'top' portfolio.

Parameters
----------
data : pl.DataFrame
    Data frame containing portfolio returns. Must include columns
    for the portfolio identifier, date, and return measurements
    (as specified in 'data_options').
direction : str, default 'top_minus_bottom'
    Direction of the long-short return calculation. Must be either
    'top_minus_bottom' or 'bottom_minus_top'. If set to
    'bottom_minus_top', the return is computed as (bottom - top).
data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'date' element
    specifies the date column, the 'ret_excess' element specifies
    the excess return column, and 'portfolio' specifies the
    assigned portfolio. Uses the 'data_options' defaults when
    None: 'date' -> 'date', 'ret_excess' -> 'ret_excess', and
    'portfolio' -> 'portfolio'.

Returns
-------
pl.DataFrame
    Data frame with columns for date and the computed long-short
    returns. The data frame is arranged by date and pivoted to
    have return measurement types as columns with their
    corresponding long-short returns.

Examples
--------
```python
import datetime as dt
import numpy as np
import polars as pl
from tidyfinance import (
    compute_portfolio_returns,
    compute_long_short_returns,
    breakpoint_options,
)
rng = np.random.default_rng(42)
dates = pl.date_range(
    dt.date(2020, 1, 1), dt.date(2028, 4, 1), '1mo', eager=True
)
data = pl.DataFrame({
    'permno': range(1, 101),
    'date': dates,
    'mktcap_lag': rng.uniform(100, 1000, 100),
    'ret_excess': rng.standard_normal(100),
    'size': rng.uniform(50, 150, 100),
})
portfolio_returns = compute_portfolio_returns(
    data, 'size', 'univariate',
    breakpoint_options_main=breakpoint_options(n_portfolios=5),
)
compute_long_short_returns(portfolio_returns)
```

filter_sorting_data(data: polars.dataframe.frame.DataFrame, filter_options: dict = None, data_options: dict = None, quiet: bool = False) -> polars.dataframe.frame.DataFrame

Filter sorting data.

Applies sample construction filters to a data frame before
portfolio sorting. Filters are applied in a fixed order:
financials exclusion, utilities exclusion, minimum stock price,
minimum size quantile, minimum listing age, positive book equity,
and positive earnings. An informational warning is emitted for
each filter that actually removes at least one observation.

Parameters
----------
data : pl.DataFrame
    Data frame containing the stock-level panel data to be
    filtered.
filter_options : dict, optional
    Dict produced by 'filter_options'. If None (the default), the
    defaults from 'filter_options' are used (i.e., no filters are
    applied). The accepted entries include:

    - 'exclude_financials' (bool): Whether to exclude financial
      firms (SIC codes 6000 to 6799). Defaults to False.
    - 'exclude_utilities' (bool): Whether to exclude utility firms
      (SIC codes 4900 to 4999). Defaults to False.
    - 'min_stock_price' (float, optional): Minimum stock price
      required to include an observation. None (the default)
      applies no price filter.
    - 'min_size_quantile' (float, optional): Minimum cross-sectional
      size quantile (based on lagged market cap) required to
      include an observation. None (the default) applies no size
      quantile filter. The cutoff is computed from NYSE stocks
      only; the 'exchange' column (mapped via 'data_options') must
      be present or a ValueError is raised.
    - 'min_listing_age' (float, optional): Minimum number of months
      a stock must have been listed in CRSP. None (the default)
      applies no listing age filter.
    - 'exclude_negative_book_equity' (bool): Whether to exclude
      observations with non-positive book equity. Defaults to False.
    - 'exclude_negative_earnings' (bool): Whether to exclude
      observations with non-positive earnings. Defaults to False.

data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'siccd' element
    specifies the SIC code column, 'price' specifies the (adjusted)
    price column, 'mktcap_lag' specifies the market capitalization
    column, 'date' specifies the date column, 'listing_age'
    specifies the listing age column, 'be' specifies the book
    equity column, and 'earnings' specifies the earnings column.
    Uses the 'data_options' defaults when None.
quiet : bool, default False
    Whether informational messages should be suppressed.

Returns
-------
pl.DataFrame
    Filtered data frame, preserving the class and structure of the
    input.

Examples
--------
```python
import datetime as dt
import polars as pl
from tidyfinance import filter_sorting_data, filter_options
data = pl.DataFrame({
    'permno': range(1, 6),
    'date': [dt.date(2020, 1, 1)] * 5,
    'siccd': [6100, 2000, 4950, 3000, 6500],
    'prc_adj': [5.0, 0.5, 15.0, 20.0, 10.0],
})
filter_sorting_data(
    data,
    filter_options=filter_options(
        exclude_financials=True,
        min_stock_price=1,
    ),
)
```

implement_portfolio_sort(data: polars.dataframe.frame.DataFrame, sorting_variables, sorting_method: str, portfolio_sort_options: dict, rebalancing_month: int = None, breakpoint_function_main=None, breakpoint_function_secondary=None, min_portfolio_size: int = 1, cap_weight: float = 0.8, data_options: dict = None, quiet: bool = False) -> polars.dataframe.frame.DataFrame

Implement a portfolio sort.

A convenience wrapper that combines sample construction filtering
and portfolio return computation into a single call. Equivalent to
calling 'filter_sorting_data' followed by
'compute_portfolio_returns' with the filter and breakpoint
specifications bundled in 'portfolio_sort_options'.

Parameters
----------
data : pl.DataFrame
    Data frame containing the stock-level panel data.
sorting_variables : str or list of str
    One or two column names to sort portfolios on.
sorting_method : str
    Sorting method to use. One of 'univariate',
    'bivariate-dependent', or 'bivariate-independent'.
portfolio_sort_options : dict
    Dict produced by 'portfolio_sort_options', bundling filter and
    breakpoint specifications. The accepted entries include
    'filter_options', 'breakpoint_options_main', and
    'breakpoint_options_secondary'.
rebalancing_month : int, optional
    Month in which portfolios are rebalanced annually. None (the
    default) means monthly rebalancing.
breakpoint_function_main : callable, optional
    Function used to compute breakpoints for the main sorting
    variable. Defaults to 'compute_breakpoints'.
breakpoint_function_secondary : callable, optional
    Function used to compute breakpoints for the secondary sorting
    variable. Defaults to 'compute_breakpoints'.
min_portfolio_size : int, default 1
    Minimum number of firms in the reported portfolio
    cross-section on a given date. For univariate sorts that is
    firms per portfolio-date; for bivariate sorts that is firms
    per main-portfolio-date summed across the secondary buckets.
    Cross-sections below the threshold have their returns set to
    null. Set to 0 to deactivate the check.
cap_weight : float, default 0.8
    Quantile of the cross-sectional 'mktcap_lag' distribution at
    which market capitalizations are capped per date when computing
    the capped value-weighted excess return ('ret_excess_vw_capped').
    Must be in [0, 1].
data_options : dict, optional
    Column-name mapping (see 'data_options'). All elements are
    forwarded to 'filter_sorting_data' and
    'compute_portfolio_returns'. Uses the 'data_options' defaults
    when None.
quiet : bool, default False
    Whether informational messages should be suppressed.

Returns
-------
pl.DataFrame
    Data frame of portfolio returns as returned by
    'compute_portfolio_returns'.

Examples
--------
```python
import datetime as dt
import numpy as np
import polars as pl
from tidyfinance import (
    implement_portfolio_sort,
    portfolio_sort_options,
    filter_options,
    breakpoint_options,
)
rng = np.random.default_rng(123)
dates = pl.date_range(
    dt.date(2020, 1, 1), dt.date(2028, 4, 1), '1mo', eager=True
)
data = pl.DataFrame({
    'permno': range(1, 501),
    'date': dates.to_numpy().repeat(5),
    'mktcap_lag': rng.uniform(100, 1000, 500),
    'ret_excess': rng.standard_normal(500),
    'prc_adj': rng.uniform(0.5, 50, 500),
    'size': rng.uniform(50, 150, 500),
})
implement_portfolio_sort(
    data=data,
    sorting_variables='size',
    sorting_method='univariate',
    portfolio_sort_options=portfolio_sort_options(
        filter_options=filter_options(min_stock_price=1),
        breakpoint_options_main=breakpoint_options(n_portfolios=5),
    ),
)
```


## Estimation


estimate_betas(data: polars.dataframe.frame.DataFrame, model: str, lookback, min_obs: int = None, id_col: str = 'permno') -> polars.dataframe.frame.DataFrame

Estimate rolling betas.

Estimates rolling betas for a given model using the provided data.
For each stock, the regression specified by 'model' is fit over a
rolling calendar window of length 'lookback' (e.g. '"60mo"' for
sixty months).

The estimator avoids refitting a full regression for every window.
Instead it accumulates the per-observation cross-products that
define the normal equations (the design Gram matrix 'X'X' and the
moment vector 'X'y'), takes their rolling sums via cumulative-sum
differencing, and solves the resulting small linear system once per
window. This closed-form approach follows the fast beta estimation
described at
https://www.tidy-finance.org/blog/fast-beta-estimation/ and is
considerably faster than looping rolling regressions while
returning the same coefficients.

Parameters
----------
data : pl.DataFrame
    Data frame containing the data with a date identifier (defaults
    to 'date'), a stock identifier (defaults to 'permno'), and the
    other variables used in the model.
model : str
    Formula describing the model to be estimated (e.g.,
    'ret_excess ~ mkt_excess + hml + smb'). An intercept is
    included unless the formula ends in '- 1' (or '+ 0').
lookback : str or int
    Rolling window length. Pass a duration string — a positive
    count followed by one of '"mo"' (months), '"d"' (days),
    '"h"', '"m"', '"s"' — to roll over calendar periods, e.g.
    '"60mo"'. The window
    for a period 'v' spans every observation falling in the
    'lookback' periods ending at 'v', so gaps in a stock's history
    consume window space, and one row is returned per stock and
    period. Sub-day units require a datetime column.

    Passing a plain integer selects the legacy behaviour — a window
    of that many consecutive observations, one output row per input
    row — and emits a 'DeprecationWarning'.
min_obs : int, optional
    Minimum number of observations required to estimate the model.
    Defaults to 'round(0.8 * lookback)'.
id_col : str, default 'permno'
    Column name representing the stock identifier.

Returns
-------
pl.DataFrame
    Data frame with the estimated betas for each stock and time
    period. Contains the stock identifier and the 'date' column,
    followed by one column per model term: an 'intercept' column
    (when the model includes one) and one 'beta_<variable>' column
    per regressor.

    With a calendar 'lookback' there is one row per stock and
    period, 'date' is floored to the start of the period, and
    windows with fewer than 'min_obs' observations are dropped
    from the output. With the deprecated
    integer 'lookback' there is one row per input row, 'date' is
    the observation's own date, and sub-'min_obs' windows yield
    null coefficients.

Examples
--------
```python
from datetime import date
import numpy as np
import polars as pl
from tidyfinance import estimate_betas
rng = np.random.default_rng(1234)
dates = pl.date_range(
    date(2020, 1, 1), date(2020, 12, 1), "1mo", eager=True
)
data_monthly = pl.DataFrame({
    'date': np.repeat(dates.to_numpy(), 50),
    'permno': np.tile(np.arange(1, 51), 12),
    'ret_excess': rng.normal(0, 0.1, 600),
    'mkt_excess': rng.normal(0, 0.1, 600),
    'smb': rng.normal(0, 0.1, 600),
    'hml': rng.normal(0, 0.1, 600),
})
estimate_betas(data_monthly, 'ret_excess ~ mkt_excess', lookback='3mo')
```

estimate_fama_macbeth(data: polars.dataframe.frame.DataFrame, model: str, vcov: str = 'newey-west', vcov_options: dict | None = None, date_col: str = 'date', data_options: dict | None = None, detail: bool = False) -> polars.dataframe.frame.DataFrame | dict

Estimate Fama-MacBeth regressions.

Runs one cross-sectional ordinary least squares regression per period
of 'date_col', then averages the per-period coefficients to obtain
risk premia and aggregates them into a single tidy frame.

Parameters
----------
data : pl.DataFrame
    Panel containing the dependent and independent variables named in
    'model' plus a column with the time index. Each (date, unit)
    combination should appear at most once.
model : str
    Formula describing the cross-sectional regression
    (e.g., 'ret_excess ~ beta + bm + log_mktcap'). Standard
    formulaic syntax; an intercept is included unless the formula
    ends in '- 1'.
vcov : {'iid', 'newey-west'}, default 'newey-west'
    Standard error treatment for the time-series average of period
    coefficients. 'iid' assumes independent and identically distributed
    errors across periods. 'newey-west' applies Newey-West
    heteroskedasticity- and autocorrelation-consistent standard errors
    with Bartlett kernel.
vcov_options : dict, optional
    Tuning options for the Newey-West estimator. Recognized keys:

    - 'lag' : int, optional
        Bartlett truncation lag. If None (the default), the
        automatic bandwidth from Newey & West (1994) is used.
    - 'prewhite' : int, default 1
        Order of the VAR prewhitening filter applied before
        computing the long-run variance. Pass 0 to disable.
    - 'adjust' : bool, default False
        Apply a finite-sample degrees-of-freedom correction.
    - 'maxlags' : int, optional
        Deprecated alias for 'lag' (with 'prewhite' defaulting
        to 0). Emits a DeprecationWarning.
date_col : str, default 'date'
    Column in 'data' identifying the time index for cross-sectional
    regressions.
data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'date' element is
    used to specify the date column, overriding 'date_col'. Uses
    the 'data_options' default when None: 'date' -> 'date'.
detail : bool, default False
    If 'False' (default), return only the coefficient estimates. If
    'True', return a dict with two keys: 'coefficients' (the usual
    estimates data frame) and 'summary_statistics' (a one-row data
    frame with the average cross-sectional R-squared, adjusted
    R-squared, and number of observations per cross-section).

Returns
-------
pl.DataFrame or dict
    If 'detail' is 'False' (default), a data frame with one row per
    term in 'model', in model-term order (the intercept first, then
    the regressors as they appear in the formula), with columns:

    - 'factor' : term name ('intercept' or a regressor)
    - 'risk_premium' : time-series mean of cross-sectional coefficients
    - 'n' : number of periods used
    - 'standard_error' : SE of the time-series mean under 'vcov'
    - 't_statistic' : risk_premium / standard_error

    If 'detail' is 'True', a dict with two elements:

    - 'coefficients' : the same data frame described above
    - 'summary_statistics' : a one-row data frame with 'r_squared'
      (mean cross-sectional R-squared), 'adj_r_squared' (mean
      cross-sectional adjusted R-squared), and 'n_obs' (mean
      cross-sectional observation count)

Raises
------
ValueError
    If 'vcov' is not 'iid' or 'newey-west', if 'vcov_options'
    contains an unrecognized key, if 'date_col' is missing from
    'data', or if any date grouping has too few rows to estimate
    the cross-sectional coefficients (each grouping needs more
    rows than the number of variables in 'model').

References
----------
Fama, E. F., and MacBeth, J. D. (1973). Risk, return, and equilibrium:
Empirical tests. Journal of Political Economy, 81(3), 607-636.
https://doi.org/10.1086/260061

Newey, W. K., and West, K. D. (1987). A simple, positive
semi-definite, heteroskedasticity and autocorrelation consistent
covariance matrix. Econometrica, 55(3), 703-708.
https://doi.org/10.2307/1913610

Newey, W. K., and West, K. D. (1994). Automatic lag selection in
covariance matrix estimation. Review of Economic Studies, 61(4),
631-653. https://doi.org/10.2307/2297912

Examples
--------
```python
from datetime import date
import numpy as np
import polars as pl
from tidyfinance import estimate_fama_macbeth
rng = np.random.default_rng(1234)
dates = pl.date_range(
    date(2020, 1, 1), date(2020, 12, 1), "1mo", eager=True
)
data = pl.DataFrame({
    'date': np.repeat(dates.to_numpy(), 50),
    'permno': np.tile(np.arange(1, 51), 12),
    'ret_excess': rng.normal(0, 0.1, 600),
    'beta': rng.normal(1, 0.2, 600),
    'bm': rng.normal(0.5, 0.1, 600),
    'log_mktcap': rng.normal(10, 1, 600),
})
result = estimate_fama_macbeth(data, 'ret_excess ~ beta+bm+log_mktcap')
# Override the Newey-West settings
result_iid = estimate_fama_macbeth(
    data,
    'ret_excess ~ beta + bm + log_mktcap',
    vcov='iid',
)
# Return detailed output including R-squared and observation counts
result_detail = estimate_fama_macbeth(
    data,
    'ret_excess ~ beta + bm + log_mktcap',
    detail=True,
)
```

estimate_model(data: polars.dataframe.frame.DataFrame, model: str, min_obs: int = 1, output='coefficients')

Estimate a linear model.

Estimates a linear model specified by one or more independent
variables. It checks for the presence of the specified independent
variables in the dataset and whether the dataset has a sufficient
number of observations. Depending on the 'output' parameter, it
returns the model's coefficients, t-statistics, residuals, or any
combination in a named dict.

Parameters
----------
data : pl.DataFrame
    Data frame containing the dependent variable and one or more
    independent variables.
model : str
    Formula string describing the model to be estimated (e.g.,
    'ret_excess ~ mkt_excess + hml + smb'). Use 'y ~ x - 1' for
    no-intercept models.
min_obs : int, default 1
    Minimum number of observations required to estimate the model.
output : str or list of str, default 'coefficients'
    What to return. Must contain one or more of 'coefficients',
    'residuals', and 'tstats'. If a single value is provided, the
    corresponding object is returned directly. If multiple values
    are provided, a dict is returned.

Returns
-------
pl.DataFrame, np.ndarray, or dict
    If 'output' contains a single value: a data frame of
    coefficients or t-statistics, or a numeric vector of
    residuals. If 'output' contains multiple values: a dict with
    the requested elements. Coefficients and t-statistics are
    returned as one-row data frames with column names corresponding
    to the model terms (all-null when there are not enough
    observations). Residuals are returned as a numeric vector of
    length 'len(data)' with NaN for rows with missing data or
    insufficient observations.

Examples
--------
```python
import numpy as np
import polars as pl
from tidyfinance import estimate_model
rng = np.random.default_rng(42)
data = pl.DataFrame({
    'ret_excess': rng.standard_normal(100),
    'mkt_excess': rng.standard_normal(100),
    'smb': rng.standard_normal(100),
    'hml': rng.standard_normal(100),
})
# Estimate model with a single independent variable
estimate_model(data, 'ret_excess ~ mkt_excess')
# Estimate model with multiple independent variables
estimate_model(data, 'ret_excess ~ mkt_excess + smb + hml')
# Estimate model without intercept
estimate_model(data, 'ret_excess ~ mkt_excess - 1')
# Calculate residuals
estimate_model(
    data, 'ret_excess ~ mkt_excess + smb + hml',
    output='residuals',
)
# Return t-statistics
estimate_model(
    data, 'ret_excess ~ mkt_excess + smb + hml',
    output='tstats',
)
# Return coefficients, t-statistics, and residuals
estimate_model(
    data, 'ret_excess ~ mkt_excess + smb + hml',
    output=['coefficients', 'tstats', 'residuals'],
)
```


## Rolling and lagging


get_backend() -> str

Return the active data frame backend ('"pandas"' or '"polars"').

add_lagged_columns(data: polars.dataframe.frame.DataFrame, cols: list[str] | str, lag, max_lag=None, by: list[str] | str | None = None, drop_na: bool = False, ff_adjustment: bool = False, date_col: str = 'date', data_options: dict | None = None) -> polars.dataframe.frame.DataFrame

Append lagged columns to a data frame via a join-based approach.

When 'lag == max_lag' (the default), an equi-join is used: source
dates are shifted forward by 'lag' and matched exactly. When
'lag < max_lag', an inequality join is used: for each row, the most
recent source value within the window '[date - max_lag, date - lag]'
is selected.

The combination of 'by' and the date column must be unique in 'data'.
If 'by' is None, dates alone must be unique.

Parameters
----------
data : pl.DataFrame
    DataFrame containing the variables to lag. The date column must
    be of dtype 'pl.Date' or 'pl.Datetime'.
cols : list of str or str
    Names of the columns to lag. Each column produces a new column
    suffixed with '_lag'.
lag : int, str, datetime.timedelta, or pd.DateOffset
    Minimum lag (inclusive) to apply, e.g. '"1mo"'. An int is
    interpreted as days; strings are polars offset strings.
max_lag : int, str, datetime.timedelta, or pd.DateOffset, optional
    Maximum lag (inclusive) to apply. Defaults to 'lag' (exact lag).
by : list of str or str, optional
    Grouping column(s) (e.g. a stock identifier). Lagged values are
    matched within groups. Defaults to None.
drop_na : bool, optional
    If True, missing values in the source columns are excluded
    before matching, so the lookup skips over missing observations.
    Applied independently per column. Defaults to False.
ff_adjustment : bool, optional
    If True, only the last observation per year (within each group
    defined by 'by') is retained as a source for lagged values,
    following Fama-French conventions for annual accounting data.
    Defaults to False.
date_col : str, optional
    Name of the date column. Defaults to 'date'.
data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'date' element is
    used to specify the date column. Uses the 'data_options' default
    when None: 'date' -> 'date'.

Returns
-------
pl.DataFrame
    Data frame with the same rows as 'data' and new columns
    appended, each suffixed with '_lag'. Unmatched rows receive
    null in the lagged columns.

Examples
--------
```python
import numpy as np
import polars as pl
import datetime as dt
from tidyfinance import add_lagged_columns
rng = np.random.default_rng(42)
dates = pl.date_range(
    dt.date(2023, 1, 1), dt.date(2023, 10, 1), "1mo", eager=True
)
data = pl.DataFrame({
    'permno': [1] * 10 + [2] * 10,
    'date': dates.to_list() * 2,
    'size': rng.uniform(100, 200, 20),
    'bm': rng.uniform(0.5, 1.5, 20),
})
# Exact lag: each row gets the value from exactly 2 months earlier
add_lagged_columns(data, cols=['size', 'bm'], lag='2mo', by='permno')
# Window lag: most recent value from 2 to 4 months earlier
add_lagged_columns(
    data, cols='size', lag='2mo', max_lag='4mo', by='permno'
)
```

join_lagged_values(original_data: polars.dataframe.frame.DataFrame, new_data: polars.dataframe.frame.DataFrame, id_keys: list[str] | str, min_lag, max_lag, ff_adjustment: bool = False, date_col: str = 'date', data_options: dict | None = None) -> polars.dataframe.frame.DataFrame

Join lagged values of variables over a date range.

Joins lagged values of selected variables from one data frame
('new_data') into another ('original_data'), based on date ranges
defined by 'min_lag' and 'max_lag'. Unlike 'add_lagged_columns',
this function supports joining across data frames with different
date grids (e.g. monthly source data into quarterly target data).
All columns in 'new_data' besides 'id_keys' and the date column are
lagged and joined under their original names.

Parameters
----------
original_data : pl.DataFrame
    Target panel data. The date column must be of dtype 'pl.Date'
    or 'pl.Datetime'.
new_data : pl.DataFrame
    Source variables to lag and merge. All columns besides
    'id_keys' and the date column will be lagged and joined.
id_keys : list of str or str
    Identifier column(s) shared by both frames.
min_lag : int, str, datetime.timedelta, or pd.DateOffset
    Lower lag bound (inclusive).
max_lag : int, str, datetime.timedelta, or pd.DateOffset
    Upper lag bound (inclusive).
ff_adjustment : bool, optional
    If True, keeps only the last observation per identifier and
    year in 'new_data' before lagging (Fama-French convention).
    Defaults to False.
date_col : str, optional
    Name of the date column. Defaults to 'date'.
data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'date' element is
    used to identify the date column. Uses the 'data_options'
    default when None: 'date' -> 'date'.

Returns
-------
pl.DataFrame
    'original_data' with all columns from 'new_data' appended as
    lagged values (keeping their original names).

Examples
--------
```python
import numpy as np
import polars as pl
import datetime as dt
from tidyfinance import join_lagged_values
rng = np.random.default_rng(42)
dates = pl.date_range(
    dt.date(2020, 1, 1), dt.date(2020, 6, 1), "1mo", eager=True
)
df1 = pl.DataFrame({
    'id': [1] * 6 + [2] * 6,
    'date': dates.to_list() * 2,
})
df2 = df1.with_columns(
    x=pl.Series(rng.standard_normal(len(df1)))
)
join_lagged_values(
    original_data=df1,
    new_data=df2,
    id_keys='id',
    min_lag='1mo',
    max_lag='3mo',
)
```

compute_rolling_value(data: polars.dataframe.frame.DataFrame, f, period: str = 'month', periods: int = 12, min_obs: int = None, data_options: dict = None) -> numpy.ndarray

Compute a rolling value by period.

Applies an arbitrary summary function over rolling time-period
windows. Each window spans 'periods' units of 'period' (e.g., 12
months). Before calling 'f', rows with any missing values are
dropped from the window; if fewer than 'min_obs' rows remain, the
result is NaN instead.

Parameters
----------
data : pl.DataFrame
    Data frame with a date column named according to
    'data_options[date]' (default 'date'). The column must be of
    dtype 'pl.Date' or 'pl.Datetime'.
f : callable
    Function applied to each window. Receives the window slice
    (complete cases only) as a data frame of the active backend
    type ('pd.DataFrame' under the default 'pandas' backend,
    'pl.DataFrame' under the 'polars' backend) and must return
    a single scalar value.
period : str, default 'month'
    Calendar period unit for the rolling windows. One of 'month',
    'quarter', or 'year'.
periods : int, default 12
    Number of periods to include in the rolling window.
min_obs : int, optional
    Minimum number of non-missing rows required per window.
    Defaults to 'periods'.
data_options : dict, optional
    Column-name mapping (see 'data_options'). The 'date' element
    is used to specify the date column. Uses the 'data_options'
    default when None: 'date' -> 'date'.

Returns
-------
np.ndarray
    Numeric vector aligned with the rows of 'data'.

Examples
--------
```python
import numpy as np
import polars as pl
import datetime as dt
from tidyfinance import compute_rolling_value
rng = np.random.default_rng(42)
df = pl.DataFrame({
    'date': pl.date_range(
        dt.date(2020, 1, 1), dt.date(2021, 12, 1), '1mo', eager=True
    ),
    'value': rng.standard_normal(24),
})
df = df.with_columns(
    rolling_sd=pl.Series(compute_rolling_value(
        df,
        f=lambda x: x['value'].std(ddof=1),
        period='month',
        periods=4,
        min_obs=2,
    ))
)
```


## Supported datasets


list_supported_datasets(domain: 'Optional[str | list[str]]' = None, as_vector: 'bool' = False) -> "'pl.DataFrame | list[str]'"

List all datasets supported by 'download_data'.

Aggregates the Global Q, Fama-French, Goyal-Welch, WRDS, Pseudo Data,
and "other" tables into a single :class:'polars.DataFrame'. The legacy
Fama-French table is intentionally excluded from the master listing.

Parameters
----------
domain : str or list of str, optional
    Restrict the result to one or more domain labels (for example
    'WRDS' or '['Fama-French', 'Global Q']').
as_vector : bool, default False
    When 'True', return a list of 'type' strings instead of a
    DataFrame.

Returns
-------
polars.DataFrame or list of str
    Either a DataFrame with columns 'type', 'dataset_name', and
    'domain', or a list of 'type' strings when 'as_vector=True'.

Examples
--------
```python
from tidyfinance import list_supported_datasets
list_supported_datasets()
list_supported_datasets(domain='WRDS')
list_supported_datasets(as_vector=True)
```


## Utilities


create_summary_statistics(data: polars.dataframe.frame.DataFrame, variables: list, by: str = None, detail: bool = False, drop_na: bool = False) -> polars.dataframe.frame.DataFrame

Create summary statistics for specified variables.

Computes a set of summary statistics for numeric and boolean
variables in a data frame. It allows users to select specific
variables for summarization and can calculate statistics for the
whole dataset or within groups specified by the 'by' argument.
Additional detail levels for quantiles can be included.

The function first checks that all specified variables are of a
numeric dtype (int, float, or bool). If any variables fail this
check, a 'ValueError' is raised listing the offending columns.
Boolean columns are summarized as their numeric equivalent — for
example, the 'mean' of a boolean column is the proportion of True.

The basic set of summary statistics includes the count of non-null
values ('n'), 'mean', standard deviation ('sd'), minimum ('min'),
median ('q50'), and maximum ('max'). If 'detail' is True, the
function also computes the 1st, 5th, 10th, 25th, 75th, 90th, 95th,
and 99th percentiles ('q01' through 'q99'). Statistics are computed
for the whole dataset, or separately for each group when 'by' is
supplied.

Parameters
----------
data : pl.DataFrame
    Data frame containing the variables to be summarized.
variables : list of str
    List of column names in the data frame to summarize. These
    variables must be of a numeric dtype (int, float, or bool).
by : str, optional
    Column name to group the data before summarizing. If None (the
    default), summary statistics are computed across all
    observations.
detail : bool, default False
    Whether to compute detailed summary statistics, including
    additional quantiles. When False, computes basic statistics
    (n, mean, sd, min, q50, max). When True, additional quantiles
    (q01, q05, q10, q25, q75, q90, q95, q99) are computed.
drop_na : bool, default False
    Whether to drop missing values for each variable before
    summarizing.

Returns
-------
pl.DataFrame
    Data frame with summary statistics for each selected variable.
    If 'by' is specified, the output includes the grouping variable
    as well. Each row represents a variable (and a group if 'by' is
    used), and each column contains the computed statistics.

Examples
--------
```python
import polars as pl
from tidyfinance import create_summary_statistics
data = pl.DataFrame({
    'ret': [0.01, -0.02, 0.03, None, 0.005],
    'size': [100, 200, 150, 300, 250],
    'group': ['A', 'A', 'B', 'B', 'A'],
})
# Basic summary across all observations
create_summary_statistics(data, ['ret', 'size'])
# Grouped summary
create_summary_statistics(data, ['ret', 'size'], by='group')
# Detailed quantiles
create_summary_statistics(data, ['ret'], detail=True)
```

list_supported_indexes() -> polars.dataframe.frame.DataFrame

Return a DataFrame of supported financial indexes.

Each row corresponds to one index and pairs the index name with the
URL of an iShares CSV holdings file plus the number of header rows
to skip when parsing it. This table powers
'download_data_constituents'.

Returns
-------
polars.DataFrame
    A DataFrame with three columns:

    - 'index': name of the financial index (e.g. 'DAX', 'S&P 500').
    - 'url': URL of the CSV file with constituent holdings.
    - 'skip': number of preamble rows in the provider file. Kept for
      compatibility; the header row is now detected automatically.

Examples
--------
```python
from tidyfinance import list_supported_indexes
supported_indexes = list_supported_indexes()
print(supported_indexes)
```

list_supported_jkp_factors(region: str = None, dataset: str = 'factors') -> polars.dataframe.frame.DataFrame

Return the regions and factors supported by Global Factor Data.

Queries the live availability manifest of
`Global Factor Data <https://jkpfactors.com/data>`_ and returns the
regions and selectors that can be passed to 'download_data_jkp'.

Parameters
----------
region : str, optional
    A region or country code. If provided, the function returns
    the selectors (factor codes, or industry classifications when
    'dataset="industry"') available for that region. If 'None'
    (the default), it returns the available region codes.
dataset : str, default 'factors'
    The Global Factor Data product to query, one of 'factors',
    'portfolios', or 'industry'.

Returns
-------
pl.DataFrame
    When 'region' is 'None', a data frame with a single 'region'
    column listing the available region codes. When 'region' is
    provided, a data frame with a 'region' column and a 'factor'
    column listing the selectors (factor codes, or industry
    classifications when 'dataset="industry"') available for that
    region.

Examples
--------
```python
from tidyfinance import list_supported_jkp_factors
list_supported_jkp_factors()
list_supported_jkp_factors('usa')
list_supported_jkp_factors('usa', dataset='portfolios')
```

list_tidy_finance_chapters() -> list

Return the chapter slugs of the Tidy Finance book.

Provides the URL-safe identifiers used to address chapters on the
Tidy Finance website. The list can be passed to
'open_tidy_finance_website' or formatted into other links.

Returns
-------
list of str
    Chapter slugs in the order they appear in the book. Each slug
    corresponds to a chapter page on https://www.tidy-finance.org.

Examples
--------
```python
from tidyfinance import list_tidy_finance_chapters
list_tidy_finance_chapters()
```

open_tidy_finance_website(chapter: str = None) -> None

Open the Tidy Finance website or a specific chapter.

If 'chapter' is omitted, the main landing page of the Python
edition of Tidy Finance is opened. Otherwise the URL for the
requested chapter is constructed and opened. An unknown chapter
name falls back to the main page.

Parameters
----------
chapter : str, optional
    Slug of the chapter to open (e.g. 'beta-estimation'). Must
    match an entry returned by 'list_tidy_finance_chapters'.

Returns
-------
None
    The function is called for its side effect of launching the
    default browser at the appropriate URL.

Examples
--------
```python
from tidyfinance import open_tidy_finance_website
open_tidy_finance_website()
open_tidy_finance_website('beta-estimation')
```

winsorize(x, cut: float) -> numpy.ndarray

Winsorize a numeric vector at symmetric quantiles.

Replaces values below the lower 'cut' quantile and above the upper
'1 - cut' quantile with the corresponding quantile boundaries. The
length of the vector is preserved.

Parameters
----------
x : numpy.ndarray or polars.Series
    A numeric vector to winsorize. Inputs that are not already
    arrays are coerced via 'numpy.asarray'.
cut : float
    Proportion of observations replaced at each tail. For example,
    'cut=0.05' clips the lowest and highest five percent. Must lie
    in '[0, 0.5]'.

Returns
-------
numpy.ndarray
    Vector with the same length as 'x' in which extreme values have
    been replaced by the lower and upper quantile cutoffs.

Examples
--------
```python
import numpy as np
from tidyfinance import winsorize
rng = np.random.default_rng(123)
data = rng.standard_normal(100)
winsorized = winsorize(data, 0.05)
```

trim(x, cut: float) -> numpy.ndarray

Trim a numeric vector by removing extreme observations.

Drops values below the lower 'cut' quantile and above the upper
'1 - cut' quantile. The returned vector is therefore shorter than
the input.

Parameters
----------
x : numpy.ndarray or polars.Series
    A numeric vector to trim.
cut : float
    Proportion of observations removed at each tail. For example,
    'cut=0.05' removes the lowest and highest five percent. Must
    lie in '[0, 0.5]'.

Returns
-------
numpy.ndarray
    Vector with the extreme observations removed.

Examples
--------
```python
import numpy as np
from tidyfinance import trim
rng = np.random.default_rng(123)
data = rng.standard_normal(100)
trimmed = trim(data, 0.05)
```


## Backend


set_backend(backend: str) -> None

Set the global data frame backend for the tidyfinance API.

Parameters
----------
backend : str
    Either 'pandas' (the default) or 'polars'.

Raises
------
ValueError
    If 'backend' is not a recognized value.
ImportError
    If 'backend' is 'pandas' but the 'pandas' package is not
    installed.

Notes
-----
The package computes in polars internally. The pandas backend wraps
the public API at the package boundary: pandas inputs are converted
to polars before each call, and polars outputs are converted to
pandas on return. The polars backend is a pass-through with zero
conversion overhead.

On conversion from pandas, known calendar-date columns (e.g.
'date', 'datadate', 'trd_exctn_dt') are cast from
'polars.Datetime' to 'polars.Date', since pandas has no plain date
dtype. Any time-of-day component in a column with one of these
names is therefore dropped on input. Timezone-aware datetime
columns are never cast.

get_backend() -> str

Return the active data frame backend ('"pandas"' or '"polars"').