Introduction

TBD

Baremetal and Cloud Resources

Nordix uses few tenants OpenStack public clouds for most of the development activities.

Apart from the cloud resources, work is ongoing to bring up a baremetal lab to ensure we can provide baremetal resources to community as well when needed.

OpenStack Tenants

Access to Project Instances

The projects that are developed on Nordix OpenStack Tenants have instances for the development work. (ONAP and Acumos are the 2 examples) SSH access to project instances are provided on a need basis and this access must be under the control of the Infra Core Team.

All the project instances have jumphost between them and the internet. The access to jumphost must be requested on discuss maillist and is given by the Infra Core Team. The only access mechanism that is accepted is key-based authentication so requestors must provide their ssh public keys.

Once the access is given to the jumphost instance, the developer then needs to contact to the project team to get access to the rest of the project instances. Infra Core Team does not have any responsibility over how the project utilizes the project instances or who gets access to them.

Accounts on jumphost can be created using regular Linux commands. Here are the steps.

• Login to jumphost.<project> as a user with sudo rights
• Create user using regular linux commands
  • sudo useradd -G <project group> -m -s /bin/bash <username> # usernames are generally constructed using the first letter of the first name and the last name
• Copy user's ssh public key into  /home/<username>/.ssh/authorized_keys
• Update /etc/ssh/sshd_config and append username into AllowUsers
• Restart sshd service

Gerrit

TBD

Setting Up a New Gerrit Based Upstream Project

Setting up a new Gerrit based upstream project requires following certain steps and reaching out to communities for getting additional rights for Nordix CI/CD.

Most of the steps are same no matter what upstream project is setup within Nordix except the differences highlighted in corresponding step.

Some of the steps can be performed in advance and some others must be done after the others are completed.

Here are the steps to follow.

• Create an account on SSOs of upstream projects
  
  • Common for all upstream  SSOs:
    • A user must be created or an existing user must be used if a project from a certain community has already been mirrored in Nordix Gerrit.
    • User name for Nordix Infra is nordix and this user name must be used while creating a new account if it is available. If it is not available, it must be discussed first so everyone knows the username.
    • The mail address for nordix user is infra@nordix.org and the people who work with Nordix Infra is subscribed to this mailbox.
  • LF:
    • LF hosted projects use LFID and once an account is setup for Nordix Infra user there, same account can and should be used for accessing all the systems of all the LF hosted projects including but not limited to Gerrit, Jenkins and so on.
    • Please note that an account for Nordix Infra named nordix has already been created on LFID so you should not create a new account.
  • OpenDev:
    • OpenDev uses Ubuntu One and an account for Nordix Infra named nordix must be created.
• Set username on Upstream Project's Gerrits
  • Username to use is nordix.
  • This is done in Profile tab on Gerrit Settings page after the initial login to upstream Gerrit. Please note that this is a one time operation and username can not be changed afterwards.
• Upload ssh public key for the user on Upstream Project's Gerrits
  • This is done on SSH Public Keys tab on Gerrit Settings page after the initial login to upstream Gerrit.

  • The key to use must be nordix user's key and not random as it is used for cloning repos and so on and configured on Nordix Jenkins.

    ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCdyLRXDIU4w/0H/kZa+2Fw/NLPTyW83F9cQqrahGybbff/pY3CxBKUqRefIp6SLjcR5TjrWOLVO6hlan+nzUzVahlgim8YIGYiD7l6ZuWBvlt/WpqqZOswRLKsgQgTDBUNFXl4V++bsZfbJwlv1QnNaeXGYdfGulXTnQ0wk4+/rBhPXqKVIPJyjnnrz1yCxwyRGOtB9hzSiG1VoKEoddgqIydRnxPQER7K5mc3E6CmvNr8FB5sbo+urT8EVLlb9Df8a7G0XRfMrM0z+1mFjlXG6ckvbIrlfJjQkeK00cXiFwwAmRQeHKZEQrl9++uhjcueIT0TYnGMW7ZW/Hl6NfV8z21pcj+erD3ltDXn3yZjlzf44ekvtNGegJ7hlB7mSdY17rk46QPSlPtPDeibZFj33c+jhZVFeB8PV7DOfXhqtNEVCZ/GlajQlgqbJqy1ILVftD/AhZpyEK0P6j8RiP7IF+rKVrdLsnoXPuHPMV+l+JKJE7STDHiCST0Opn5+WbQhiysDkORCfZMa+q02+/V4X6Khc97ws2LVJU19EQpqGSTYRew44E/cPGFMd3MvlyvzULV56XEtGESAYGAJdgj8g+zif8R4HjfgXo0KRgnz9yk3UU9qo2QZ1aqfu7+mO5r33cVk5ZVUIp3r0f7SD6OEOpj2nlc+cIaV6XRlF/BDYQ== infra@nordix.org

• Manual Verification

  • Attempt cloning a repo using SSH from Upstream Gerrit with the username nordix and its ssh private key to ensure username, ssh keys and other stuff is correctly setup.

• Create Gerrit Server Configuration on Nordix Jenkins

  • This is done on Nordix Jenkins Gerrit Trigger Configuration page by clicking Add New Server link.

  • Fill in the name of the new Gerrit Server to Add New Server field. Format of the server names is <Upstream Project> Gerrit such as ONAP Gerrit, OpenDev Gerrit.

  • Clicking Copy Existing Server Configurations will make configuration easier and reduce the chance to make mistakes as fields will be prepopulated with the values. Update the fields specific to the project you are setting up.

  • After you save the configuration for the new Gerrit Server, you should click the red ball and see it turning blue. If it stays red, something might be wrong either with the configuration in Nordix or in upstream Gerrit. Troubleshooting is needed.
• Ask for Gerrit Stream Event Rights from upstream community
  
  • OpenDev:
    • OpenDev generally allows stream rights for all the users by default so nothing extra needs to be done for it in upstream. Just verification is needed in Nordix Jenkins by creating trigger macros and setting a job for a mirrored project and watching it getting triggered by different Gerrit Events such as change-merged. See the next step.
    • If it doesn't work, upstream community needs to be reached in #openstack-infra channel. Ask others who know who to talk in OpenStack community for help to reduce the time it takes to get this fixed.
  • LF:
    • LF does not allow stream rights for all the users by default and this must be requested using corresponding project's helpdesk.
    • To make things faster, ask the people who know who to talk to in LF for help to reduce the time it takes to get stream rights for nordix granted.
• Add Trigger Macros for new Gerrit Server into JJB Macros in nordix infra/cicd repo
  • See existing macros and copy/paste/adjust them.
• Setup build server(s) for the project
  • If the developers need special build servers, they need to be setup.
  • Otherwise, existing build servers should be ok to use.
• Ensure Gerrit Host Keys are added to build servers
  
  • This can perhaps be automated but the easiest way to do this is to login to build servers and clone one of the repos from upstream project via ssh and accept the key.
  • Not doing this will cause job build failures with ssh key complaints.

• Mirror projects

  • See the next chapter

Mirroring a Project from Upstream Gerrit

Mirroring a project from upstream Gerrit can be done by following the steps listed below.

• SSH to Nordix Gerrit server as infra user
• clone the repo of the project that is about to be mirrored
• ensure all the remote branches are pulled
• create the project on Nordix Gerrit
• copy cloned repository folder into the newly created Gerrit projects root folder
• assign the appropriate access control permissions to the mirrored project
• flush Gerrit cache

Once the above steps are done without issues, the mirrored project should be visible on Nordix Gerrit Web Interface and its parent should reflect to what you set.

If the project is visible there, further work is needed to be done to ensure the project repo is kept up to date with the upstream and push upstream jobs are created on Nordix Jenkins in order to support development workflow.

• create rebase and push-upstream jobs using jjb
• run rebase job manually to ensure any new change that might have been merged in upstream is synched to Nordix

Following example demonstrates these steps by mirroring a project from ONAP Gerrit into Nordix Gerrit.

ssh infra@gerrit.nordix
git clone --bare https://gerrit.onap.org/r/testsuite.git $HOME/repos/onap/testsuite
ssh -p 29418 infra@gerrit.nordix.org gerrit create-project testsuite
cp -rf $HOME/repos/onap/testsuite/* /data/gerrit/git/testsuite.git/
ssh -p 29418 infra@gerrit.nordix.org gerrit set-project-parent --parent infra/acl/onap-global testsuite
ssh -p 29418 infra@gerrit.nordix.org gerrit flush-caches --all

See this change for creating rebase and push-upstream jobs on Jenkins.

Adding branch from upstream to existing project in Gerrit

Currently only the branches defined in the Jenkins rebase job will be synchronized with Nordix Gerrit. Please follow steps below to add a branch from upstream to existing project in Nordix Gerrit:

1. Update the rebase,  push-upstream and verify (if exist) Jenkins job with the new branch. Please see this change for an example.
2. Execute the rebase Jenkins job for given project to pull down the branch to Nordix Gerrit.

Managing Gerrit Access

If a project inherits rights from a certain project, the ACL for the project can be managed by modifying corresponding files in the project such as groups and project.config which the rights are inherited from.

As an example, default All-Projects project will be used to list the steps so it is important to adjust the project name depending on what you are doing.

mkdir All-Projects && cd All-Projects
git init
git remote add origin ssh://fdegir@gerrit.nordix.org:29418/All-Projects.git
git fetch origin refs/meta/config:refs/remotes/origin/meta/config
git checkout meta/config
# edit files groups and project.config depending on what is done
git commit -a -m "Modified groups and project.config"
git push origin meta/config:meta/config

Nordix Gerrit has few parent projects created in order to control access to various projects on Nordix Gerrit.
Below table lists the projects and their purposes so if a project you are creating or mirroring fits into one of these, choose that one.
If you don't see an appropriate parent project there, you can create it but please take a discussion with the rest of the Infra Team before doing this.

On top of having parent projects to inherit access rights from, Gerrit has User Groups as well.
Below table lists the groups and their purposes so if a user you are granting rights for fits into one of these, choose that one.
If you don't see an appropriate group there, you can create it but please take a discussion with the rest of the Infra Team before doing this.

JIRA

TBD

Jenkins

TBD

Artifact Repositories

Nordix provides various artifact repository services depending on the type of artifacts.

• Artifactory: General purpose artifact repository manager which can be used for storing ISOs, RPMs, QCOW2 images and so on.
• Harbor: Private container image registry to store images built within Nordix or mirror images that are consumed from upstream communities.

Artifactory

TBD

Harbor

Primary mechanism Nordix employed to control access is as below.

• Anonymous pulls are enabled.
• Self-registration is enabled and anyone who registers will end up in readers group which has the same access rights as Anonymous users.
• Users that are member of the group builders can pull and push images.
• Users that are member of the group owners own specific repositories and have pull, push rights and rights to alter the repo structure. (TODO: find out if project specific owner groups can be done or if we should have owner groups per repo)
• Users that are member of the group admin have full admin rights over all repos and harbor itself.

Admin Contacts for the Services and Tools