Migrate source code from SVN to GitHub repository

[jmckenna]

• must bring in all SVN log history

[gfenoy]

During the discussion that happened on IRC today, @omshinde has pointed out this relevant link.

I tried using the svn2git command line tool and get some kind of result.

The command used was the following:

mkdir GITFROMSVN
cd GITFROMSVN
svn2git http://svn.zoo-project.org/svn/ --verbose --tags tags --metadata --authors ../authors.txt 
git branch -m master main

The authors.txt file used in the command line was created bu using the informations provided on developper name and email.

The result is the following:

• Branches:

  PublicaMundi_David-devel
  PublicaMundi_David_integration-devel
  PublicaMundi_David_integration_01-devel
  branch-1.2
  branch-1.3
  branch-1.4
  branch-1.5
  branch-1.6
  branch-1.7
  branch-1.8
* main
  ms-style
  prototype-v0

• Tags:

rel-1.2.0
rel-1.2.0-rc1
rel-1.2.0-rc2
rel-1.2.0-rc3
rel-1.3.0
rel-1.3.0-rc1
rel-1.3.0-rc2
rel-1.4.0
rel-1.4.0-rc1
rel-1.5.0
rel-1.5.0-rc1
rel-1.6.0
rel-1.6.0-rc1
rel-1.6.0-rc2
rel-1.7.0
rel-1.7.0-rc1
rel-1.8.0
rel-1.8.0-rc1
zoo-project-1.2.0

• Logs: (short extract)

commit c3b5331182dcf5346c7b45123022e2ccc78acb04
Author: Gérald Fenoy <gerald.fenoy@geolabs.fr>
Date:   Wed Jun 29 11:21:51 2011 +0000

    Fix FastCGI support. Small fix to store mapon disk and manage session. Fix typo in main.cfg.
    
    git-svn-id: http://svn.zoo-project.org/svn/trunk@253 4b6b6ad6-68b3-4dcc-9231-bd9b83bc5dc3

From my understanding it looks like a starting point.

Obviously, all the referenced trac issues cannot be pointed automatically. But this may come later on I guess.

What do other think of trying to now create a repository named "ZOO-Project" within the "ZOO-Project" organization to see how it looks like?

Indeed, from now on, I am not sure how the published result will look like. I mean, for instance I don't know if all the user like mine, where djay (SVN user) was translated to an existing GitHub handle (Gérald Fenoy gerald.fenoy@geolabs.fr, will get the corresponding GitHub account shown online.

I would be in for trying to create the new repository. Waiting for your feedbacks.

[jmckenna]

I think we should create a new repository "ZOO-Project" here, to try the svn2git process.

By the way, the GRASS team did really well to document and record all of their scripts (https://courses.neteler.org/remarks-on-svn-trac-to-github-migration/), including:

• their svn2git changes: https://trac.osgeo.org/grass/browser/grass-addons/tools/svn2git/
• their trac2github code: https://github.com/landam/trac2github/tree/grass (converts milestones/tickets/comments/labels & Trac usernames)

[omshinde]

That is an amazing beginning.
+1 from my side.
I would be very happy to contribute to any of the associated tasks. Thanks!

[kalxas]

Looks good, +1

[kalxas]

Can you please provide a link to the authors.txt file so I can add my info?

[gfenoy]

The authors.txt file can now be edited from the repo (authors.txt).

[kalxas]

Thanks, my info is correct.

[gfenoy]

As a first tentative, I have tried to push the GIT repository built using the svn2git tool on ZOO-Project-svn2git repository.

It looks almost fine, there seems to be an issue in the authors.txt for Markus Neteler, I probably do not set the proper email address causing the user account not to be set properly. If you look at https://github.com/ZOO-Project/ZOO-Project-svn2git/commits/main?before=42a90aebb847c7500da79449fa38f824340eba56+490&branch=main and search for "Commits on Dec 7, 2011" you will see that the Markus Neteler is written in the same way as if no GitHub account was corresponding.

Anyway, for a first try, I think the result is promising.

[gfenoy]

From my last test with the ZOO-Project-svn2git repository, I can confirm that we can preserve link between SVN users and GitHub accounts.

Now we should know if we should wait for the trac issues to be ported to GitHub issues to get the link between the trac issue and the commit message, in case it is referring any. There is another option, we may link to the current trac issue for every issue mentioned in commit messages.

Ideas are welcome.

[venka-foss4g]

Hi Gerald, Jeff and Rajat, Thanks for all the hard work with the migration to github. Let me know if I can help in anyway. My knowledge about github migration is rudimentary and I may not be able to come up with ideas to make your task easier. Best Venka

…

On 3/30/2021 1:04 AM, Gérald FENOY wrote: From my last test with the ZOO-Project-svn2git repository, I can confirm that we can preserve link between SVN users and GitHub accounts. Now we should know if we should wait for the trac issues to be ported to GitHub issues to get the link between the trac issue and the commit message, in case it is referring any. There is another option, we may link to the current trac issue for every issue mentioned in commit messages. Ideas are welcome.

[gfenoy]

To clarify the situation.

In the SVN commit message we can reference a Trac ticket by using the #XXX syntax where XXXis the id of the Trac ticket. For instance, if you look at this page, in the commit message the corresponding Trac Ticket (#175) is referenced.

So for me, we have two options to solve this link issue between commit messages and Trac tickets that should become GitHub issue.

The first option is to update the commit messages to replace the #XXX by [TracTicket#XXX](URL/To/Trac/Ticket). This option implies that we keep the Trac tickets online read-only. Also, we may imagine referring to the corresponding up-to-date GitHub issues link within the ticket.

The second option is to wait for the Trac tickets to be moved to GitHub issues and replace the references to Trac tickets with references to GitHub issues.

I would personally prefer the first option.

[gfenoy]

I decided to create the ZOO-Project GitHub repository.

Currently, there is still the issue with ticket link. Maybe we can simply specify somewhere (wiki or anywhere else) that the reference in commit message containing "#XXX" prior to 9fb0a48dd5a95d6b6ac0c0a8367192e888b50cc5 are referring to https://zoo-project/trac/ticket/XXX .

In any case, commit messages can be updated afterward.

Should we consider that from now, we consider this new ZOO-Project GitHub repository as our official source code location?

[jmckenna]

@venka-foss4g thanks for your support! Note: similar to my "good question" response, to acknowledge the question but to give a response for things that I have no idea about, ha, you/we can use the Github emoji (click the smiley icon above each of these comments here on the github page) to give a +1 to show that you're following, you support it, and, it shows that you/we are being active ha. win-win ha.)

[jmckenna]

@gfenoy I think the in-line ticket references in comments (for others reading who are new to Github, I'm referring to https://docs.github.com/en/github/writing-on-github/autolinked-references-and-urls ) must be rewritten to point to the associated Github ticket (goal is to not have any references remaining to Trac). To be more clear: when I mentioned to keep Trac tickets as read-only, I meant that only for archival purposes, I don't think any of our new infrastructure should point to any Trac page anymore, that's the goal. To discuss together tomorrow in meeting. (this could also likely be where we 'earn our money', meaning it may be a custom script)

[jmckenna]

Should we consider that from now, we consider this new ZOO-Project GitHub repository as our official source code location?

Yes, as the sooner we do this, the sooner Google search bots archive our repository, making it easier for users to find our code. Again, should discuss this as PSC tomorrow, together. PS. hope we can use our Github wiki for the meeting agenda tomorrow ;) hint.

[gfenoy]

The commit messages referring any ticket were fixed by using the following command and the rewrite.py file available on this repo (ref. https://lists.osgeo.org/pipermail/gdal-dev/2018-March/048246.html):

git clone https://github.com/ZOO-Project/ZOO-Project --mirror
git filter-branch -f --msg-filter 'python3 /Users/geraldfenoy/MapMint/GIT2020/rewrite.py' --tag-name-filter cat  -- --all 
git push --mirror

Now for every ticket referred from a commit message, the link corresponding to the current ticket on the trac system is properly provided.

Once tickets will be migrated, we may consider changing the commit messages again to update the reference.

[gfenoy]

As discussed during today PSC meeting, we can now close this ticket as SVN history is now in good shape on the ZOO-Project/ZOO-Project repository.

Also, I would like to inform every @ZOO-Project/developer that from now on, we should consider the ZOO-Project/ZOO-Project repository as our current source versioning system.

The SVN will not be used anymore.