Gerrit

GitHub Replication Configuration

Initial configuration (required once)

  1. Hiera configuration:

    Gerrit::extra_configs:
      replication_config:
        config_file: '/opt/gerrit/etc/replication.config'
        mode: '0644'
        options:
          'remote.github':
            # ORG == the Org on GitHub
            # ${name} is literal and should exist in that format
            url: 'git@github.com/ORG/${name}.git'
            push:
              - '+refs/heads/*:refs/heads/*'
              - '+refs/heads/*:refs/tags/*'
            timeout: '5'
            threads: '5'
            authGroup: 'GitHub Replication'
            remoteNameStyle: 'dash'
    
  2. If a $PROJECT-github account does not exist on GitHub, create it, setup 2-factor authentication on the account, and add the recovery tokens to LastPass. The email for the account should be to collab-it+$PROJECT-github@linuxfoundation.org

  3. Copy the public SSH key for the ‘gerrit’ user into the GitHub account

  4. On the Gerrit Server do the following:

    # create 'root' shell
    sudo -i
    # create 'gerrit' shell
    sudo -iu gerrit
    # Add the server key to gerrit's known_hosts file
    ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
    # exit from 'gerrit' shell
    exit
    # restart Gerrit so that SSH changes are properly picked up
    systemctl restart gerrit
    # exit from 'root' shell
    exit
    
  5. Add the account to the GitHub Organization as a Member

  6. Configure the Organization with the following options:

    1. Members cannot create repositories

    2. Members cannot delete or transfer repositories

    3. Set the default repository permission to Read

    4. Require 2FA (Two Factor Authentication) for everyone

  7. Create a Replication team in the organization and add the $PROJECT-github account

  8. In Gerrit create a ‘GitHub Replication’ group that is empty

  9. Set the following ACL on the All-Projects repository

    refs/*
      Read
        DENY: GitHub Replication
    

Repository replication setup (repeat for each repository)

Note

After initial setups, descibed above gerrit project creation, github repo creation and gerrit replication are now done with lftools commands.

To create_repo, clone_repo, create_groups_file and add_gitreview:

lftools gerrit create [OPTIONS] GERRIT_URL LDAP_GROUP REPO USER

To create a github repo:

lftools github create-repo --sparse ORGANIZATION REPOSITORY DESCRIPTION

To enable replication:

lftools gerrit create --enable GERRIT_URL LDAP_GROUP REPO USER

Manual Process

Perform the following in each repository mirrored from Gerrit

  1. Create the repository in the GitHub organization replacing any occurrence of ‘/’ with ‘-’ as ‘/’ is an illegal character for GitHub repositories.

  2. Add the Replication Team to the repository with write privileges

  3. In Gerrit add the following ACL

    refs/*
      Read
        ALLOW: GitHub Replication
    
  4. Perform initial code drop

    The initial code drop must be present before you enable Gerrit replication for a repository.

  5. Enable repo replication

    To enable replication for a single repo:

    ssh -p 29418 ${youruid}@${project_gerrit} replication start --wait --url ${repo_url}
    

    To enable replication for more than one repo:

    ssh -p 29418 ${youruid}@${project_gerrit} replication start --all --wait
    
  6. Watch GitHub to see if the repo starts to replicate, if not troubleshoot by looking at ~gerrit/logs/replication*

Gerrit Prolog Filter

LF has automated the handling of committers and creation of repos, which makes it crucial that the INFO.yaml file are correct.

To enforce this, Gerrit needs to do some extra checks on the submitted files, in particular the INFO.yaml file. The change set can not have more than 1 file, if the file is INFO.yaml, to enable fault tracing and handling.

To summarize, below are the requirements:

  1. Ensure that self review with +2 is not allowed.

  2. Ensure that INFO.yaml has been automatically reviewed and approved by Jenkins.

  3. Ensure that INFO.yaml file is alone in the change set.

A gerrit prolog filter, located in the Gerrit All-Projects repository, implements the above requirements. The project repos inherits and apply the filter.

Note

For further information about Prolog and Gerrit refer to the Prolog Cookbook

Below are the instructions on how to install this filter.

  1. Clone the project’s All-Projects repo

    git clone "ssh://<user>@gerrit.<project>.org:29418/All-Projects"
    git fetch origin refs/meta/config:config
    git checkout config
    
  2. Confirm rules.pl is not modified.

    Verify that the rules.pl file are either missing, or contains code for non-author-approval like below

    submit_filter(In, Out) :-
         In =.. [submit | Ls],
         add_non_author_approval(Ls, R),
         Out =.. [submit | R].
    
    add_non_author_approval(S1, S2) :-
         gerrit:commit_author(A),
         gerrit:commit_label(label('Code-Review', 2), R),
         R \= A, !,
         S2 = [label('Non-Author-Code-Review', ok(R)) | S1].
    add_non_author_approval(S1, [label('Non-Author-Code-Review', need(_)) | S1]).
    

    Note

    If rules.pl contains something else, please confirm before continuing, since below steps will overwrite the old rules.pl.

  3. Get the user id for the automatic codereview users.

    • Go to the appropriate Gerrit’s groups page (https://gerrit.example.org/r/admin/groups)

    • Click on Non-Interactive Users

    • Click on Members Verify these users are the correct ones. For ONAP that would be ONAP Jobbuilder, ecomp jobbuilder, and LF Jenkins CI

    • Click on Audit Log Find the Added row for each user. The member column contains the userid (in parentheses). For instance, for ONAP Jobbuilder the record states Added ONAP Jobbuilder(459) where the user id is 459.

    These userid’s should replace the userid’s in the rules.pl further down in this document. Below is the relevant code area in rules.pl.

    % Define who is the special Jenkins user
    jenkins_user(user(459)).   % onap-jobbuilder@jenkins.onap.org
    jenkins_user(user(3)).     % ecomp-jobbuilder@jenkins.openecomp.org
    jenkins_user(user(4937)).  % releng+lf-jobbuilder@linuxfoundation.org
    
  4. Replace/Create rules.pl with below content

    # Start ignoring allow_passive_voice

    submit_filter(In, Out) :-
         In =.. [submit | Ls],
         % add the non-owner code review requiremet
         reject_self_review(Ls, R1),
         % Reject if multiple files and one is INFO.yaml
         ensure_info_file_is_only_file(R1, R2),
         % Reject if not INFO file has been verified by Jenkins
         if_info_file_require_jenkins_plus_1(R2, R),
         Out =.. [submit | R].
    
    % =============
    %filter to require all projects to have a code-reviewer other than the owner
    % =============
    reject_self_review(S1, S2) :-
         % set O to be the change owner
         gerrit:change_owner(O),
         % find a +2 code review, if it exists, and set R to be the reviewer
         gerrit:commit_label(label('Code-Review', 2), R),
         % if there is a +2 review from someone other than the owner,
         % then the filter has no work to do, assign S2 to S1
         R \= O, !,
         % the cut (!) predicate prevents further rules from being consulted
         S2 = S1.
    
    reject_self_review(S1, S2) :-
         % set O to be the change owner
         gerrit:change_owner(O),
         % find a +2 code review, if it exists, and set R to be the reviewer
         gerrit:commit_label(label('Code-Review', 2), R),
         R = O, !,
         % if there is not a +2 from someone else (above rule),
         % and there is a +2 from the owner, reject with a self-reviewed label
         S2 = [label('Self-Reviewed', reject(O))|S1].
    
    % if the above two rules did not make it to the ! predicate,
    % there are not any +2s so let the default rules through unfiltered
    reject_self_review(S1, S1).
    
    
    % =============
    % Filter to require one file to be uploaded, if file is INFO.yaml
    % =============
    ensure_info_file_is_only_file(S1, S2) :-
         % Ask how many files changed
         gerrit:commit_stats(ModifiedFiles, _, _),
         % Check if more than 1 file has changed
         ModifiedFiles > 1,
         % Check if one file name is INFO.yaml
         gerrit:commit_delta('^INFO.yaml$'),
         % If above two statements are true, give the cut (!) predicate.
         !,
         %set O to be the change owner
         gerrit:change_owner(O),
         % If you reached here, then reject with Label.
         S2 = [label('INFO-File-Not-Alone', reject(O))|S1].
    
    ensure_info_file_is_only_file(S1, S1).
    
    
    % =============
    % Filter to require approved jenkins user to give +1 if INFO file
    % =============
    % Define who is the special Jenkins user
    jenkins_user(user(459)).   % onap-jobbuilder@jenkins.onap.org
    jenkins_user(user(3)).     % ecomp-jobbuilder@jenkins.openecomp.org
    jenkins_user(user(4937)).  % releng+lf-jobbuilder@linuxfoundation.org
    
    
    is_it_only_INFO_file() :-
         % Ask how many files changed
         gerrit:commit_stats(ModifiedFiles, _, _),
         % Check that only 1 file is changed
         ModifiedFiles = 1,
         % Check if changed file name is INFO.yaml
         gerrit:commit_delta('^INFO.yaml$').
    
    if_info_file_require_jenkins_plus_1(S1, S2) :-
         % Check if only INFO file is changed.
         is_it_only_INFO_file(),
         % Check that Verified is set to +1
         gerrit:commit_label(label('Verified', 1), U),
         % Confirm correct user gave the +1
         jenkins_user(U),
         !,
         %set O to be the change owner
         gerrit:change_owner(O),
         % Jenkins has verified file.
         S2 = [label('Verified-By-Jenkins', ok(O))|S1].
    
    if_info_file_require_jenkins_plus_1(S1, S2) :-
         % Check if only INFO file is changed.
         is_it_only_INFO_file(),
         % Check if Verified failed (-1) +1
         gerrit:commit_label(label('Verified', -1), U),
         % Confirm correct user gave the -1
         jenkins_user(U),
         !,
         % set O to be the change owner
         gerrit:change_owner(O),
         % Jenkins failed verifying file.
         S2 = [label('Verified-By-Jenkins', reject(O))|S1].
    
    if_info_file_require_jenkins_plus_1(S1, S2) :-
         % Check if only INFO file is changed.
         is_it_only_INFO_file(),
         !,
         % set O to be the change owner
         gerrit:change_owner(O),
         S2 = [label('Verified-By-Jenkins', need(O))|S1].
    
    if_info_file_require_jenkins_plus_1(S1, S1).
    
  5. Push it to Gerrit

    git add rules.pl
    git commit -m "LF initial prolog filter"
    git push origin HEAD:refs/meta/config
    

    # Stop ignoring