[p-sawicki]

Add CI validation similar to mypy primer for mypyc changes. The new job compiles and tests configured projects in PRs that change mypyc. Initially configure black as there have been regressions in mypyc in the past that broke it.

The mypyc_primer script clones the project, sets it up and builds it according to configuration using mypyc pinned at head_sha (PR commit in CI), and runs configured tests. If any step fails, the run is repeated but this time using mypyc pinned at base_sha (target branch commit in CI) so failures introduced by the mypyc change can be distinguished from a broken upstream project or infra issues.

Once results for all projects are ready another CI job posts a comment in the PR either with a success message or a list of failures with links to logs. Example comments are in my fork: success and failure. If the job is skipped because mypyc is not changed in the PR the comment job is skipped as well - example.

The mypyc_primer script can also be used to validate changes locally. Instructions for this are in the comment at the top of the script.

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[ilevkivskyi]

Thanks! I will try to look in more details in next few days.

Also cc @hauntsaninja in case you would prefer to incorporate this into mypy_primer instead.

[ilevkivskyi]

Not sure about this. Should we set this to 2 (or whatever is the equivalent of -g)?