Use of CircleCI "Github App" -intrinsic authentication, https: ...not SSD or Oauth or Deploy keys

[rgleason]

Some wise guy is likely to tell me I am way behind the times here.
Just stop before saying anything further and think about
"Why wasn't he made aware of this change? Perhaps I should have advised him and others using OpenCPN!"

Secondly this is not fully detailed or described. I just discovered it today and ran a successful build of Findit_pi using the
CircleCI Github App after adding the Circleci Github App to my Circleci account, and then adding and configuring a new Pipeline in Findit which uses the Github App. I then manually triggered a build using the Github App (which has intrinsic authentication). I believe the use of https:// for download is perfectly acceptable and does not then open the door to unwanted users triggering the builds and deployments.

The use of the Github App eliminates:

• Need for separate Oauth Keys for each plugin or use of SSH,
• Simplifies CI Setup for new developers because they just have to configure their Cloudsmith Deployment key.
• Requires changing circleci/config.yml and ci scripts to use https:// to download repos.
• This is actually quite easy if you use a validator and a command such as https-checkout

commands:
    https-checkout:
      steps:
        - checkout:
            clone:
              ssh: false
    deploy-code:
        parameters:
            install_python:
                type: boolean
                default: false
            deploy_use_orb:
                type: boolean
                default: true

.

1. Use of the CircleCI Github App is recommended over using OAuth.
2. Old Github repos are likely to be using Oauth for authentication which requires a separate key for each plugin, and uses SSH git@github.com to download the repo.
3. Old Github repos can enable the newer CircleCI Github App and run separate pipelines using the Github App.
4. These pipelines are best used with config.yml scripts that call for https:// to download the repo from github therefore the script may require adjustments. The findit_pi config-https.yml is an example that works.
   6.Keeping the old Oauth circleci pipeline is perfectly ok.
5. I have only manually triggered the build from within my Circleci account. Circleci Github App has this ability now.
6. There are other techniques to trigger it to build from github.

The build is here https://github.com/rgleason/FindIt_pi The two android did not build so it shows X. All the other builds worked.
After installing Github App in your Circleci Account, you then add a new pipeline in the plugin you wish to build.

I will show some screen shots which may be helpful. It is NOT a full description however.

VCS Integration https://circleci.com/docs/guides/integration/using-the-circleci-github-app-in-an-oauth-org/

https://circleci.com/docs/guides/integration/using-the-circleci-github-app-in-an-oauth-org/

https://circleci.com/docs/guides/integration/oss/

https://circleci.com/docs/guides/orchestrate/triggers-overview/#run-a-pipeline-from-the-circleci-web-app

[rgleason]

How to add or check the CircleCI "Github App"

CircleCI → Organization Settings → VCS Connections

1. Open CircleCI Organization Settings (The GitHub App can only be installed from inside CircleCI, not from GitHub.)
2. In CircleCI → Organization Settings- Make sure you are viewing the correct organization. You must have admin permissions in both GitHub and CircleCI
3. Go to VCS Connections - This is where CircleCI exposes the GitHub App installation entry point.
4. Organization Settings → VCS Connections- You should see GitHub App listed
5. Click “Install” or “Configure” This launches the GitHub App installation flow inside GitHub.

• If you see Install, you have not connected the GitHub App yet
• If you see Configure, the app is installed but not fully set up
• If you do NOT see it, your account may be in legacy OAuth-only mode

6. Select Repositories to Grant Access

• The GitHub App uses fine‑grained permissions and short‑lived tokens.
• Choose All repositories or Only selected repositories
• Ensure the repo you want CircleCI to build is checked
• Approve the permissions GitHub requests

7. Return to CircleCI to Complete Setup

• GitHub should redirect you back to CircleCI automatically.
• CircleCI will confirm the installation
• You can now enable pipelines for your repos
• GitHub App pipelines unlock new features and better security

8. GitHub App works alongside OAuth
   If your CircleCI org URL contains /github/ or /gh/, you already have the legacy OAuth integration.
   The GitHub App is an add‑on that provides:

• Fine‑grained permissions
• Short‑lived tokens
• Access to new pipeline features

Next steps

1. Change the circleci/config.yml to use https:// to download the github repo.
2. Add a new Pipeline (select Github App)

[jongough]

I have tried to follow this, but I get stuck about half way through step 7. I cannot see/find anything about 'enable pipelines' or what features are unlocked. At step 8. the only way to use https is to specify the url, including the user name, in the config.yml file which makes it unique to each repository being used. This has the knock on effect of requiring changes to the config.yml be implemented by each plugin rather than just copying the current version to get the latest builds.

I have done the change to the config.yml file to try and use https, but it still appears to use ssh. The use of ssh: false is an invalid statement in the yaml file. Again I am lost at step 2 of the next steps.

This process has to work for all builds making sure that only the CMakeLists.txt file is the only 'user' required changed file. All the other stuff is done to simplify, for the plugin developer, the multiplatform build process.

[rgleason]

Did you manage to add the "Github App" to you main CircleCI account/organization?

CircleCI → Organization Settings → VCS Connections -->Find GitHub App listed Click “Install” or “Configure”

After that you go to one of your organizatins CircleCI Plugins and "Add a new Pipeline" and configure it to use the "Github App" See the two screenshots above, notice the top arrow points to a pipeline named "Github App" which was just added, and the bottom arrow points to another older pipeline named "Github Oauth.

Then I was able to "trigger" Findit manually using the Github App pipeline.

That is what I did to run Findit_pi, just as an example, along with the changes to config.yml

============================

[rgleason]

Jon regarding

I have done the change to the config.yml file to try and use https, but it still appears to use ssh.

• This may be because you have not installed the Github App or perhaps you are not using a Github App pipeline (most likely).
• I had this problem too, initially.

but it still appears to use ssh. The use of ssh: false is an invalid statement in the yaml file. Again I am lost at step 2 of the next steps.

• I believe "use of "ssh: false" is an invalid statement, is because you are still using a pipeline configured in the old way. If you use a new "Added" Pipeline that is selected for "Github App" and shows "Github App" as shown in one of the screenshots above, you will not get this statement.

The only change to the FINDIT_pi from your TP setup was creating this command and then changing all the "checkout"s to "https-checkout" in the individual builds. See The findit_pi config-https.yml is an example that works.

commands:
https-checkout:
steps:
- checkout:
clone:
ssh: false

Other than that change the name of the plugin is found normally. Therefore the TP code will work with any plugin so configured.

PS: I am no expert, but I did get Findit_pi working using the new CircleCI "Github App" and it will save Devs a lot of time setting up plugins, and it will save me time managing multiple plugins.

PS Thank you for looking at this alternative. I may make a PR. Findit_pi ran all of the builds, after a second run for the android and jammy builds.

[jongough]

I have just done a test and there appears to be no need to change anything in the config.yml file. All the changes occur in github and circleci. You need to add the github app and ensure you give it permissions to the repositories you want to work on . Then in circleci you need to add a pipeline trigger that uses the github app and disable the ssh trigger. With these two changes the checkout occurs using https and there is no modification needed to the .circleci/config.yml .

I have tried the above on both testplugin_pi and ocpn_draw_pi and it seems to work. The failure in MSVC compiles is due a change I was working on but is not relevant to this process.

[rgleason]

Jon, Thank you, these changes will be a great improvement to the ease of setup which also utilizes the most modern pipeline in circleci.

I like your improvement, taking the https command out of config and adding it to the scripts. Or is it that I must disable the Oauth pipeline to force running https?

Gradually I will drop all the old Github Oauth pipelines, and remove the auth keys in Circleci.

Then I will just need the manage and setup the Cloudsmith API Key in Circleci Environment. How simple!

When we have time we should also simplify the manual so that we have complete up to date instructions.

[rgleason]

Jon,
You should know that the reason I added a temporary directory was so that I could "disable ssh" for that directory first, then successfully run https:// Otherwise I was running into issues and failures.

Under TestPlugin Updates branch what is it that allows you to do this? Did you add code or something somewhere else? How can I be sure that CircleCI will run https:// to get the github repo?
jongough/testplugin_pi@54dd248

I see this test jongough/testplugin_pi@00cb072
where the -run mkdir empty-checkout directory and ssh:false are commented out, saving steps and time. Is this going to work in most cases? Do I have to disable all other pipelines other than the new one that I create in CircleCI for "Github App"?

commands:
    https-checkout:
      steps:
#        - run:
#            name: "Disable Cirecleci default SSH checkout"
#            command: mkdir -p /tmp/empty_checkout
        - checkout:
            clone:
#              ssh: false
    deploy-code:
        parameters:
            install_python:

Are you trying to make TestPlugin agnostic with regard to using ssh or https?
Thus we need to Enable only the exact pipeline type that we want to run?
Github App (uses https:// pipeline) OR Oauth Pipeline, (uses SSH), for example.

[rgleason]

I made these changes in Findit_pi circleci/config.yml and push and

All builds failed. https://github.com/rgleason/FindIt_pi
Why? It is trying to use SSH instead of https:// !

commands:
    https-checkout:
      steps:
#        - run:
#            name: "Disable Circleci default SSH checkout"
#            command: mkdir -p /tmp/empty_checkout
#        - checkout:
#            path: /tmp/empty_checkout
        - checkout:
            clone:
#              ssh: false

[rgleason]

Setup of CircleCI "Github App" and Pipelines and Triggers.

Summary

1. Install the new CircleCI pipeline application for Github called "Github App"
2. Configure a new pipeline using "Github App" for each plugin, so that https:// is used to download the Github Repo.
3. Remove the old pipeline using Oauth or SSH to allow the script to utilize the new "Github App" pipeline.
4. For the immediately above command to work, the OLD CIRCLECI pipeline methods must be REMOVED.

Upper Right Organizational Icon > Select User Organization > Sidebar "Org" > VCS Connections > Install the Github App

Upper Right Organizational Icon > Select User Organization > Sidebar "Org" > VCS Connections > Gear

Sidebar > Projects > [Plugin] > Pipelines

Sidebar > Projects > [Plugin] > Pipelines > Manage Triggers

Sidebar > Projects > [Plugin] > Pipelines > Manage Triggers and configure Pipelines and Triggers

Sidebar > Projects > [Plugin] > Pipelines > Add "Github App" Pipeline and Configure the Trigger to run Github App.

Sidebar > Projects > [Plugin] > Pipelines > Remove "Oauth App" Pipeline and Remove the Old Trigger.

Sidebar > Projects > [Plugin] > Pipelines > Old "Oauth App" Pipeline > Remove the Old Trigger that forces TP Plugin to run SSH and fail.

Then cleanup your plugin Github Authorizations by removing the Oauth / SSH keys.

Here are some examples.

Github > User > Plugin > Settings > Webhooks - Remove unused

Github > User > Plugin > Settings > Deploy Keys - Removed

After Circleci Setup, - from Github

Permissions showing from Github now

[rgleason]

Jon how does this look for installation and use of Circleci Github App?
We just need the information about Cloudsmith Key which I think we have done elsewhere.
Then we need the TP installation and use instructions.
We have this Outline with links, I have just added some links but it needs to be simplified and corrected.
jongough/testplugin_pi#451 (comment)