Repository navigation
Responses (Surv, Binomial, Multinomial) and the wildcard - #2
Merged
Merged
Conversation
ModelSpec(terms, response=...) and build_response(X): the response from the same data as the design, a column, a list of columns, or Surv, Binomial or Multinomial. wildcard is R's '.': the columns of X but the response, expanded at fit into expanded_terms_; wildcard(pow=2) is '.^2', exclude= is '. - x', and it can be a part of an interaction or a variable of a feature (pca(wildcard, n_components=3)). fit(X, y) checks that the indices agree. A docs page, the API reference, and tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The columns of an np.ndarray are np.int64, the terms ints: compare column identifiers by value, and expand to ints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The response of a model, and R's
., for 0.2.0.Responses.
ModelSpec(terms, response=...)andspec.build_response(X)build the response from the same data as the design. A response can be:Surv(time, status)/Surv(start, stop, status), as R's (keywordsstop=,status=,start=also work). Returns apd.DataFrameof those columns, named as inX: what glmstar'sCoxFamily(event_id=..., status_id=..., start_id=...)and lifelines take;Binomial('y')for a binary column: 0/1, boolean, or two levels, with the second level (as in R) orlevel=counted as a success.Binomial(successes, trials=...)gives either successes and failures (output='counts', R'scbind, which statsmodels'GLMtakes) or the proportion and the number of trials (output='proportion': a response and its weights, e.g. for glmstar);Multinomial('class')for the class of each row, as a categorical with the levels seen at fit or one-hot (output='onehot').Multinomial([...])takes counts.Responses learn their levels at fit, and a level unseen at fit raises. The response's columns cannot be used in a term. The response objects are fit as copies, so
responseis never modified.The method is
build_response, notresponse, becauseresponseis the parameter's name (sklearn stores parameters as attributes). The name also parallelsbuild_submodelandbuild_sequence.wildcard, an object rather than the string'.'. It stands for every column ofXexcept the response, and is expanded at fit intoexpanded_terms_;termsis unchanged.MS([wildcard], response='y')isy ~ .wildcard(pow=2)is.^2, andwildcard(exclude=['x'])is. - x(wildcard, 'g')is.:gpca(wildcard, n_components=3)is a feature built from every column. It can be tuned withterms__0__n_components, andpca()'snameis now optional.As in R's
y ~ x + ., columns (and interactions, in any order) that are already terms are not repeated.pow > 1is only allowed when the wildcard is itself a term.fit(X, y)checks that the indices agree when both are pandas objects.Tests:
tests/test_wildcard.pyandtests/test_response.py. They check against statsmodels:PHRegonSurv, and binomialGLMwith counts versus proportion plusvar_weights. The full suite gives 261 passed and 5 strict xfails, and the new tests also pass with warnings as errors.Docs: a "Responses and the wildcard" page, executed locally with Jupyter Book 2.
Sales ~ .gives the same OLS coefficients as R'slm(Sales ~ ., Carseats). The API reference gains the new classes.🤖 Generated with Claude Code