Skip to content

DRYD-2207: Add Micrometer - #584

Open
mikejritter wants to merge 14 commits into
collectionspace:developfrom
mikejritter:feature/dryd-2207-add-micrometer
Open

mikejritter wants to merge 14 commits into
collectionspace:developfrom
mikejritter:feature/dryd-2207-add-micrometer

Conversation

@mikejritter

Copy link
Copy Markdown
Contributor

What does this do?

  • Adds micrometer as a dependency
  • Adds a new JAX-RS Resource to serve Prometheus styled metrics
  • Add ResponseTimeFilter to capture response times
  • Use cspace.metrics.enabled environment variable for enabling MetricsResource

Why are we doing this? (with JIRA link)
jira: https://collectionspace.atlassian.net/browse/DRYD-2207

This is being done so that we can better track performance of CollectionSpace and see what types of issues we might be encountering. It adds various metrics (JVM, Tomcat, Response Time), so that we can a good view of the internals of the application, from heap usage, garbage collection, to the application server, and responsiveness of serving requests.

To go over some of the internals and decisions made:

  • MeterRegistryProvider - a singleton so that we create and register a MeterRegistry only a single time. It also offers convenience methods for checking if metrics were enabled, and lazily initializes the registry so that it is only created when we use it.
  • ResponeTimeFilter - a servlet filter which creates a Micrometer Timer to track how long requests take to be processed. It normalizes the request uri so that the servlet context (/cspace-services) is removed and UUIDs/refnames are replaced with templated strings to reduce cardinality.
  • MeterRegistryContextListener - on init tries to register TomcatMetrics, but does not consider this fatal if it can't. Handles closing of resources when shutting down.

Everything else I think is fairly straightforward, not too much to go over.

How should this be tested? Do these changes have associated tests?

  • In your tomcat installation's bin/setenv.sh, add -Dcspace.metrics.enabled="true" to enable metrics
  • Rebuild and start CollectionSpace
  • Test that the metrics endpoint is operational:
$ curl --user admin@core.collectionspace.org:Administrator http://localhost:8180/cspace-services/metrics
  • Test that admin access is required for the metrics endpoint
$ curl http://localhost:8180/cspace-services/metrics -w 'Response: %{response_code}\n' -s -o /dev/null

or just use --verbose

Dependencies for merging? Releasing to production?
This is built on #583, so we should review/merge that first then I can rebase so the changes are a bit smaller. I wanted to try the stacked PRs but I don't think I initially did things in the correct order (and I also don't generally use the gh tooling).

Other things we will need to do:

  • Update the dev tomcat tarball to include the new option to the CSPACE_OPTS
  • It might be needed to add a permissions toggle to the ui as well, but that depends on the hosting deployments. We should double check.
  • It might also be good to provide a sample prometheus/grafana setup. I didn't really have time but it's something I was hoping would be included. Maybe as documentation.
  • DataSource metrics should be added when we do DRYD-2208.

Has the application documentation been updated for these changes?
Documentation will need to be added for the new environment variable

Did someone actually run this code to verify it works?
@mikejritter ran locally

Have any new security vulnerabilities been handled?
n/a so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant