This repository has been archived on 2024-09-12. You can view files and clone it, but cannot push or open issues or pull requests.
svrjs-blog/source/_posts/How-to-mirror-a-self-hosted-Git-repository-to-GitHub.md

213 lines
8.6 KiB
Markdown
Raw Normal View History

---
title: How to mirror a self-hosted Git repository to GitHub?
date: 2024-04-23 00:03:26
tags:
- git
- github
category: Tips
thumbnail: /images/covers/How-to-mirror-a-self-hosted-Git-repository-to-GitHub.png
---
A mirror of self-hosted repository is helpful, in case if your self-hosted Git server goes down. GitHub is one of the many Git hosting services, and it also has an issue tracker, and supports pull requests.
This post will guide you through setting up a GitHub mirror of you self-hosted Git repository.
As an example, we will be using our Git repository at "https://git.svrjs.org/easywaf-integration.git" and our GitHub mirror at "https://github.com/svr-js/easywaf-integration"
## Importing a Git repository to GitHub
First off, you need to import a Git repository to GitHub. You can visit [the import page](https://github.com/new/import), in order to import the Git repository. First, in your GitHub dashboard click "New", then click the "Import a repository" link.
Type your Git repository clone URL in the "The URL for your source repository" field. You may need to enter credentials, if your repository is protected with username and password. Then type your repository name and choose, if the repository is public or private. Then click "Begin import".
![Importing a GitHub repository...](/images/github-mirror-import.png)
You may see a 500 Internal Server Error page while importing the repository. In this case you may need to wait, until the repository is imported. If it's not, then you may need to [contact the GitHub support](https://support.github.com/contact).
Once the repository is imported, you can copy the clone URL by clicking at "Code", then switching to "Local" and "SSH" tabs, and clicking the copy icon.
_github-mirror-copy-cloneurl.png_
## Mirroring a self-hosted Git repository to GitHub
First, change your path to a path to your Git repository, like this:
```bash
cd /var/lib/git/easywaf-integration.git
# Replace the /var/lib/git/easywaf-integration.git with path to your Git repository
```
Then, run this Git command to add a new GitHub remote:
```bash
sudo git remote add --mirror=push github git@github.com:svr-js/easywaf-integration.git
# Replace the git@github.com:svr-js/easywaf-integration.git path with path you just copied
```
Then, create a `post-receive` hook in the `hook` directory (for example using `sudo nano hooks/post-receive`) in the Git repository with these contents:
```bash
#!/bin/bash
git push --quiet --follow-tags github
```
And grant the executable permissions to the hook:
```bash
sudo chmod +x hooks/post-receive
```
Check the owner of a Git repository:
```bash
stat .
```
The output should look like this:
```
File: .
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 254,1 Inode: 528424 Links: 8
Access: (0775/drwxrwxr-x) Uid: ( 999/ svrjs) Gid: ( 999/ svrjs)
Access: 2024-04-22 22:45:42.192000000 +0200
Modify: 2024-04-22 22:45:51.364000000 +0200
Change: 2024-04-22 22:45:51.364000000 +0200
Birth: 2023-07-29 23:53:41.444000000 +0200
```
Then change the owner to avoid permission conflicts:
```bash
sudo chown -hR svrjs:svrjs .
# Replace svrjs:svrjs with user name and group you have got using `stat` command
```
Then create the `.ssh` directory, grant permissions to it and generate the SSH key for the user (if you're doing Git on SSH, then it is probably a "git" user; if you're doing Git on HTTP(S), then it is probably a "www-data" user; SVR.JS web server has "svrjs" user; you may also check the owner):
```bash
sudo mkdir svrjs:svrjs "$(eval echo ~svrjs)"/.ssh
sudo chown -hR svrjs:svrjs "$(eval echo ~svrjs)"/.ssh
sudo chmod 400 "$(eval echo ~svrjs)"/.ssh
sudo runuser -s /bin/bash -c ssh-keygen svrjs
# Replace "svrjs" with the user name
```
Then, type the contents of the SSH public key, and copy it to clipboard:
```bash
sudo cat "$(eval echo ~svrjs)"/.ssh/id_rsa.pub
# Replace "svrjs" with the user name
# You may also try id_dsa.pub file
# After executing the "cat" command, copy the contents to the clipboard.
```
In GitHub, click on the user icon, click on "Settings", click on "Authentication keys" and click on "New SSH key". Type the key title, select the "Authentication key" key type, and paste the contents of the `id_rsa.pub` or `id_dsa.pub` file.
![Adding SSH keys to GitHub...](/images/github-mirror-ssh-key.png)
Click "Add SSH key" button to add the SSH key.
On a Git server, test the SSH connection:
```bash
sudo runuser -s /bin/bash -c 'ssh -o "StrictHostKeyChecking=no" git@github.com' svrjs
# Replace "svrjs" with the user name
```
If you see this:
```
Hi [GitHub user name]! You've successfully authenticated, but GitHub does not provide shell access.
```
that means that the SSH setup is working correctly.
You can test the mirroring using:
```bash
git clone https://git.svrjs.org/easywaf-integration.git # Replace "https://git.svrjs.org/easywaf-integration.git" with Git clone URL.
cd easywaf-integration # Replace "easywaf-integration" with the repository name
echo "Test" > somefile.txt
git add .
git commit -m 'GitHub test'
rm somefile.txt
git add .
git commit -m 'GitHub test 2'
git push
```
After pushing those changes, you can check commits at the GitHub repository. If you see "GitHub test" and "GitHub test 2" commits, that means that the mirroring works properly.
![GitHub commit view](/images/github-mirror-github-commits.png)
**You have now mirrored a self-hosted Git repository to GitHub!**
## Mirroring a GitHub repository to the self-hosted Git server
You can use GitHub Actions in order to mirror a GitHub repository back to a self-hosted Git server.
First off, enable GitHub Actions by going to a GitHub repository, clicking on "Settings" tab, clicking on "Actions", then on "General", click on "Allow all actions and reusable workflows", and click on "Save" button below the radio buttons.
Then add secrets by going to "Secrets and variables", then "Actions", and clicking on "New repository secret". If you're managing secrets for an organization, click on "Manage organization secrets", and click on "New organization secret". Then type the secret name to the "Name" field and the secret contents to the "Secret" field. Click "Add secret" in order to add a secret.
If you're pushing a Git repository through HTTPS, then add a `GIT_PASSWORD` secret with password you use for pushing a Git repository.
If you're pushing a Git repository through SSH, then add a `GIT_SSH_PRIVATE_KEY` secret with a SSH private key and a `GIT_SSH_KNOWN HOSTS` secret with a output of `ssh-keyscan -H example.com` command (replace "example.com" with a hostname of a Git server).
To obtain a SSH private key, execute these commands:
```bash
ssh-keygen -f id_rsa
ssh-copy-id -i id_rsa.pub git@git.example.com
# Replace "git@git.example.com" with a username and hostname of a Git server
```
After adding secrets, go to the "Actions" tab, click on "set up a workflow yourself" and paste contents of one of two workflow files:
If using HTTP(S) (replace "https://git.svrjs.org/easywaf-integration.git" with a Git push URL, and "git" with a Git push username):
```yaml
name: Repo sync GitHub -> Self-hosted Git server
on:
push:
branches:
- '**'
jobs:
syncgit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: spyoungtech/mirror-action@v0.5.1
with:
REMOTE: "https://git.svrjs.org/easywaf-integration.git"
GIT_USERNAME: git
GIT_PASSWORD: ${{ secrets.GIT_PASSWORD }}
```
If using SSH (replace "ssh://git@example.com/example-repository.git" with a Git push URL):
```yaml
name: Repo sync GitHub -> Self-hosted Git server
on:
push:
branches:
- '**'
jobs:
syncgit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: spyoungtech/mirror-action@v0.5.1
with:
REMOTE: 'ssh://git@example.com/example-repository.git'
GIT_SSH_PRIVATE_KEY: ${{ secrets.GIT_SSH_PRIVATE_KEY }}
GIT_SSH_KNOWN_HOSTS: ${{ secrets.GIT_SSH_KNOWN_HOSTS }}
```
After pasting the contents, click "Commit changes", then click "Commit changes" again.
![Adding a GitHub Actions workflow...](/images/github-mirror-action.png)
After committing these changes, you may need to wait for changes to be pushed to a self-hosted Git repository. You can check commits on the server using:
```bash
git log
```
![The output of the `git log` command](/images/github-mirror-git-log.png)
If you see a commit message like "Create main.yml", that means, that mirroring a GitHub repository to a self-hosted Git repository is working.
**You have now mirrored a GitHub repository to the self-hosted Git server!**