[feature/caching-rework] Centralize build planning

[lbussell]

This PR centralizes build planning and policies across multiple separate commands. The design is not perfect as-is but I would like to work on improving it incrementally.

Design

The design centers around three new types: BuildGraph, BuildPlanner, and IBuildPolicy.

A BuildGraph represents the relationships between the images in the manifest.
The BuildPlanner takes the BuildGraph in combination with an IBuildPolicy and uses it to determine what actions need to be taken.

classDiagram
direction LR

class BuildCommand {
}
class GenerateBuildMatrixCommand {
}
class GetStaleImagesCommand {
}
class BuildTarget {
}
class BuildGraph {
  +Targets
  +Parents
  +Children
  +SharedBuildTargets
}
class BuildPlanner {
  +CreatePlanAsync(graph, imageInfo, policy)
}
class IBuildPolicy {
}
class BuildPlanItem {
  +Target
  +Action
  +Reasons
  +PublishedImage
}
class BuildAction {
  NoAction
  UsePublishedImage
  PublishExistingImage
  BuildImage
}
class BuildReason {
  +Message
  +Cause
}
class PublishedImage {
  +Source
  +Image
  +SharedTags
}

BuildCommand --> BuildGraph : creates
GenerateBuildMatrixCommand --> BuildGraph : creates
GetStaleImagesCommand --> BuildGraph : creates

BuildCommand --> BuildPlanner : executes build plan
GenerateBuildMatrixCommand --> BuildPlanner : uses for trimming
GetStaleImagesCommand --> BuildPlanner : checks for stale images

BuildGraph *-- BuildTarget
BuildPlanner --> BuildGraph
BuildPlanner --> IBuildPolicy : evaluates
BuildPlanner --> BuildPlanItem : produces

BuildPlanItem --> BuildTarget
BuildPlanItem --> BuildAction
BuildPlanItem *-- BuildReason
BuildPlanItem --> PublishedImage
PublishedImage --> BuildTarget : metadata source

Loading

Build policies

There are multiple "policies" that we check images against to determine what actions to take. Previously, all these checks were strewn about the codebase and not managed centrally anywhere.

The CompositeBuildPolicy combines multiple different build policies into one that we can pass to the build planner.

classDiagram
direction LR

class IBuildPolicy {
  +EvaluateAsync(context) BuildPolicyResult
}
class BuildPolicyContext {
  +BuildGraph Graph
  +BuildTarget Target
  +PublishedImages
}
class BuildPolicyResult {
  +BuildAction Action
  +BuildReason[] Reasons
}
class CompositeBuildPolicy {
}
class AlwaysBuildPolicy {
}
class MissingPublishedImagePolicy {
}
class TagSetChangedPolicy {
}
class DockerfileChangedPolicy {
}
class BaseImageChangedPolicy {
}
class ImageDigestCache {
}
class IGitService {
}

IBuildPolicy <|.. CompositeBuildPolicy
IBuildPolicy <|.. AlwaysBuildPolicy
IBuildPolicy <|.. MissingPublishedImagePolicy
IBuildPolicy <|.. TagSetChangedPolicy
IBuildPolicy <|.. DockerfileChangedPolicy
IBuildPolicy <|.. BaseImageChangedPolicy

IBuildPolicy --> BuildPolicyContext : input
IBuildPolicy --> BuildPolicyResult : output
CompositeBuildPolicy o-- IBuildPolicy : combines

BuildPolicyResult --> BuildAction
BuildPolicyResult *-- BuildReason
BuildPolicyContext --> BuildGraph
BuildPolicyContext --> BuildTarget
BuildPolicyContext --> PublishedImage

BaseImageChangedPolicy --> ImageDigestCache
DockerfileChangedPolicy --> IGitService

Loading

[mthalman]

Are we able to have some kind of common logic for constructing the composite policy so that it's shared amongst the commands? I know there are differences but the overall structure is similar.

[mthalman]

Having the enum value represent its priority is a hidden contract that can be avoided by just have an explicit Action/Priority pair instead.

[mthalman]

GetStaleImages had never checked Dockerfile commits before. This policy will cause a failure when attempting to get a Dockerfile commit because the repo clone that had been done in SubscriptionHelper deletes the clone after loading the manifest, causing the Dockerfile to no longer exist on disk.

[lbussell]

Good catch, for now the best approach is probably to only check for base image updates in GetStaleImages.

[mthalman]

Just selecting last can be misleading, right? It would seem better to determine the winning action and use its reason.