CircleCI cannot find context "dockerhub-auth"

[sajith]

In #3483, we added a "dockerhub-auth" context which holds DockerHub credentials to tahoe-lafs' CircleCI settings, in order to solve the problem imposed by DockerHub's impending rate limits. It turns out that having an organization-specific context will prevent CircleCI from running CI tasks for pull requests made by contributors who have chosen to fork tahoe-lafs repository at GitHub.

Here is an example CI run: ​https://app.circleci.com/pipelines/github/jaraco/tahoe-lafs/23/workflows/e2869d9c-43c7-4475-9aac-690826136eb7. All of those failed tasks have failed with the same "could not find context" error message.

This happens because the tahoe-lafs organization's CircleCI account has a "dockerhub-auth" context, but any other account that forked the repository won't have such a context, unless they add one at CircleCI. This is the source of those errors, and it could be pretty confusing.

A solution would be for the account that forked the repository to create a "dockerhub-auth" context (an empty context will do) under CircleCI's "organization settings".

An even better solution would be to adapt our CircleCI configuration such that it will continue to work even when a "dockerhub-auth" context is not present. In such cases, CI should continue to function, even if DockerHub rate-limits image pulls. This way new contributors won't need to have commit access to the "canonical" tahoe-lafs repository.

[exarkun]

Since pulling Docker images is a core part of CircleCI's infrastructure and not something we have much direct control over (except the knobs that they specifically expose to us) I don't know how feasible this is.

[btlogy]

Looking at the latest PRs made from forks, I'm not sure if this is still an active problem.

For instance, here are the logs of a typical "Spin up environment" step from the fork of ​LeastAuthority:

Starting container tahoelafsci/ubuntu:22.04-py3.10
Warning: No authentication provided, using CircleCI credentials for pulls from Docker Hub.
  image cache not found on this host, downloading tahoelafsci/ubuntu:22.04-py3.10
22.04-py3.10: Pulling from tahoelafsci/ubuntu
29202e855b20: Pull complete 
6b6333d5c00a: Pull complete 
ff1afc76ab31: Pull complete 
95d8bf2f10c1: Pull complete Digest: sha256:cef80d0eb6b1aecfc79bb809955e183c87bae357bfbda304dfce37343bea6964
Status: Downloaded newer image for tahoelafsci/ubuntu:22.04-py3.10

After having looked closer at the org/settings/context on CircleCI, I've discovered that LeastAuthority? also has a context named dockerhub-auth (created before the one of Thaoe-LAFS). This explain why the above works.

Also, the dockerhub-auth context of Tahoe-LAFS org. seems to be allowed to All members, Tahoe Committers, which could explain why forks from individual members of Tahoe-LAFS also works.

I tend to agree with the earlier analysis: there is no apparent way for us to implement a logic where the authentication would be skipped if the context was not accessible (in "forking orgs." settings or non-member contributor).

Though this could change if we decide to switch to an other CI (e.g.: GH), I'm closing this issue unfortunately as wontfix.

[btlogy]

Just for the sake of it, I've tested once more using an fork from an other org. (la-test), and I've not been able to re-produce this missing dockerhub-auth issue!

[btlogy]

It seems like CircleCI has changed the way the authentication works, avoiding the failure documented in the description of this issue.

Using this la-test fork, I've been able to trigger the master branch w/o having any dockerhub-auth context (unlike for LeastAuthority fork tested before), and the "Spin up environment" step shows exactly the same ​result:

Starting container tahoelafsci/ubuntu:22.04-py3.10
Warning: No authentication provided, using CircleCI credentials for pulls from Docker Hub.
  image cache not found on this host, downloading tahoelafsci/ubuntu:22.04-py3.10
22.04-py3.10: Pulling from tahoelafsci/ubuntu
29202e855b20: Pull complete 
6b6333d5c00a: Pull complete 
ff1afc76ab31: Pull complete 
95d8bf2f10c1: Pull complete Digest: sha256:cef80d0eb6b1aecfc79bb809955e183c87bae357bfbda304dfce37343bea6964
Status: Downloaded newer image for tahoelafsci/ubuntu:22.04-py3.10

In addition, the same job for the Tahoe-LAFS master branch also claims No authentication provided for Docker hub:

Starting container tahoelafsci/ubuntu:22.04-py3.10
Warning: No authentication provided, using CircleCI credentials for pulls from Docker Hub.
  image cache not found on this host, downloading docker.io/tahoelafsci/ubuntu:22.04-py3.10
29202e855b20: Download complete 
95d8bf2f10c1: Download complete 
ff1afc76ab31: Download complete 
6b6333d5c00a: Download complete 
...

This could mean:

1. the original problem has been fixed by CircleCI,
2. and/or our current credentials for hub.Docker.com have expired (see #4142)

[btlogy]

I've been playing with those context a bit further.

And because "Pass secrets to builds from forked pull requests" is disabled on CircleCI for Tahoe-LAFS, I believe that the dockerhub-auth context can only be accessible to build triggered by PR within the Tahoe-LAFS repo (from "local" branch, thus only useful for maintainers).