Add support for query tokenization parameters - #435
Draft
matt-bernhardt wants to merge 3 commits into
Draft
Conversation
❌ 6 blocking issues (7 total)
|
Coverage Report for CI Build 32425848990Coverage increased (+0.006%) to 98.333%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
![qltysh[bot]](https://avatars.githubusercontent.com/in/890766?s=60&v=4){kind=small}
| extract_tokenization_params(enhanced_query) | ||
| @query['index'] = ENV.fetch('TIMDEX_INDEX', nil) | ||
| @query['booleanType'] = enhanced_query[:booleanType] | ||
| @query.compact! |
| TimdexBase::Client.query(TimdexSearch::GeodistanceQuery, variables: query) | ||
| else | ||
| TimdexBase::Client.query(TimdexSearch::BaseQuery, variables: query) | ||
| end |
| # drop_alt = 0.4 | ||
|
|
||
| # VCR.use_cassette('default tuning for stock query') do | ||
| # get "/results?q=#{query}&semanticMustBoostThreshold=#{must_default}&semanticDropBoostThreshold=#{drop_default}&tab=timdex" |
| # end | ||
|
|
||
| # VCR.use_cassette('alternate tuning for stock query') do | ||
| # get "/results?q=#{query}&semanticMustBoostThreshold=#{must_alt}&semanticDropBoostThreshold=#{drop_alt}&tab=timdex" |
| next unless enhanced_query[tp.to_sym].present? | ||
|
|
||
| @query[tp] = coerce_to_float?(tp) ? enhanced_query[tp.to_sym]&.strip.to_f : enhanced_query[tp.to_sym]&.strip.to_i | ||
| end |
** Why are these changes being introduced: Our search API recently added support for some optional parameters that influence how search queries are handled. We can use these parameters in our query relevance measurement in order to identify the optimal values, but the UI needs to support them for that happen. A complication is that these parameters are not included in the public schema for the API. ** Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/use-636 ** How does this address that need: This adds support for these three tuning parameters to the UI. It does this by building a separate pathway for querying the GraphQL API that is separate from the graphql-client gem. The gem rejects these parameters because they are not in the API's public schema, so we fall back to Net::HTTP for this pathway. ** Document any side effects to this change: Implementing this feature in this way will mean that we need to keep more things in sync as we continue to develop the application.
With the decision to make tuning parameters a disclosed option, we no longer need to rely on a separate querying pathway in order to use them.
matt-bernhardt
force-pushed
the
use-636
branch
from
August 19, 2026 15:17
77244ec to
f480785
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds support for some query tuning parameters to the search UI. These are not intended for human use, and are marked as experimental in the API documentation, but because our relevance measurement workflow connects to the UI application, we need to add support for them here in some fashion.
Side effect
The application builds links between tabbed displays using the parameters which are supplied - but only based on an allow list. This PR does not currently add these tuning parameters to that allow list, so any human user who tries to supply them will need to continually re-add them to the querystring rather than reyling on clicking around the interface.
I believe this is acceptable, because these parameters are intended for use by our relevance workflow - which adds them automatically without relying on links.
The parameters not being on the allow list, however, also means that when they're present the application logs will include warnings about unpermitted parameters as each tab link is rendered. The messages will look like
Unpermitted parameters: semanticMustBoostThreshold, semanticDropBoostThreshold, semanticShortQueryMaxTokensif all three parameters are supplied - and this message will appear once for each tab in the application.Ticket
https://mitlibraries.atlassian.net/browse/use-636
Developer
Accessibility
New ENV
Approval beyond code review
Additional context needed to review
E.g., if the PR includes updated dependencies and/or data
migration, or how to confirm the feature is working.
Code Reviewer
Code
added technical debt.
Documentation
(not just this pull request message).
Testing